How to Split Up an Existing Unity Git Project into Smaller Unity Packages

Update 4/29/2020: indicated new dependencies added in Template Unity Package under Step 8 that should be left in the asmdef files.

After years of making a ton of Unity games, I’ve accumulated a lot of tools that are gathered into a single Template Unity Project that I start every game jam project in. That said, the project is now much larger than I would like. Some tools, such as the web host domain list, is only applicable for projects intended to be exported to the web. In addition, since everything is built on the same project solution, compiling new scripts takes longer than necessary. So I figured it was time to split up this mammoth project into smaller Unity packages. But how?

True to Unity’s theme of democratizing game development, they have extensive documentation on how to utilize the new Unity Package Manager to better import other tools and projects. Furthermore, mob-sakai has an excellent tutorial on how to setup a Git repository with the command, git subtree split, such that the Package Manager can easily import the project. But all these information are only great for creating a single Unity Package; what if, like me, you have a sprawling Unity project that you want to split up into multiple, smaller Unity Packages? After doing some research, I’ve came to the conclusion that the best method is to use Unity’s local package directory structure, and more importantly, git submodule add to nest another Git project to easily drag-and-drop project files into. This post will go into more details on how this works.

Note: the example I used in the instructions below can be found in this Github repository: github.com/OmiyaGames/omiya-games-cryptography. And hey, help me keep making games, tools, and tutorials like the one below by supporting my studio on Ko-fi:

Table of Content

1. Preparations
2. Directions
   1. Clone your Unity project (if you haven’t already)
   2. Setup a new repo for the new package
   3. Clone the new package with git submodule add
   4. Update package.json via text editor
   5. Update all *.asmdef files via text editor
   6. Generate meta files with Unity
   7. Update package.json with Unity
   8. Update all *.asmdef files with Unity
      1. (Optional) push changes to the submodule repository
   9. Move the project files to the appropriate package folder
      1. (Optional) Troubleshooting dependencies
      2. (Optional) Troubleshooting Editor namespace conflict
      3. (Optional) Renaming elements
      4. (Optional) Deleting unnecessary folders
   10. Move the sample assets to the appropriate package folder
   11. Update Markdown files
   12. Tag and push all changes
3. Verification
4. (Optional) Adding package to OpenUPM
   1. Fork the OpenUPM repository
   2. Fill out the submission form
   3. Create a new YAML file
   4. Merge the file to OpenUPM
   5. Verification

Preparations

This post assumes the reader is somewhat familiar with Git version control — either via TortoiseGit, or through the command line interface (instructions will be provided for both) — and the JSON file format, which Unity uses to store display info about your package. It is highly recommended to review the purpose of each of these tools if you don’t know about them already, before reading further.

Second, this post assumes the reader already has a Unity project that is hosted on an Git online repository, and like me, planning to split it up into smaller packages. It’s highly recommended to backup this repository, just in case: I personally forked my project to make sure any changes I make remain separate from the original project. Since the Unity package manager accepts URLs to open-source Git projects, we’ll be working on creating a number of new Git projects, each representing a new Unity package. I’ll be using Github and some of its web interface, but in theory, any repository, including your own self-hosted repositories, should work just as well with only a few minor changes.

Directions

1. Clone your Unity project (if you haven’t already)

Start by having your large project git cloned onto your own local computer. For this post, I’ll be using the aforementioned forked Template Unity Project. To do this with TortoiseGit, just right click an empty folder, and select “Git Clone…” Follow the instructions on the wizard to clone the repo locally.

The command line version is pretty simple, of course:

# If there are no submodules in the Git repository
git clone [Git URL] [Folder]

# If there are submodules, remember to add --recurse-submodule
git clone --recurse-submodules [Git URL] [Folder]

End result: you should have the project available on your computer

2. Setup a new repo for the new package

Next, we create a new repository hosted separately from the original project with the file names following the Unity package folder structure, copied below:

• package.json
• README.md
• CHANGELOG.md
• LICENSE.md
• Editor
  • Unity.[YourPackageName].Editor.asmdef
  • EditorExample.cs
• Runtime
  • Unity.[YourPackageName].asmdef
  • RuntimeExample.cs
• Tests
  • Editor
    • Unity.[YourPackageName].Editor.Tests.asmdef
    • EditorExampleTest.cs
  • Runtime
    • Unity.[YourPackageName].Tests.asmdef
    • RuntimeExampleTest.cs
• Documentation~
  • [YourPackageName].md

To make this process easier, I’ve created the Template Unity Project on Github. If you were planning on hosting your package on Github, just login to the site and click on the “Use this Template” button to spin up the new repository.

For this tutorial, I’ll be creating a package called “String Cryptographer,” using the above template as basis.

If you plan on hosting the package elsewhere (e.g. Bitbucket), simply download a copy of the ZIP file, extract it, and upload those files into the new repository.

3. Clone the new package with git submodule add

Return back to your local copy of the project, and navigate into the Packages folder.

If you’re using TortoiseGit, right-click the folder to bring up the context menu, and select “TortoiseGit -> Submodule Add…”

Then fill in the URL to the repository, and the name of the folder to put the package in. Note the folder is based off of the root folder, so don’t forget to add Packages folder as part of the path field:

The equivalent operation on command line is as follows:

# Assuming we're already at the root folder of the project
git submodule add [Git URL] Packages/[Folder]

This should create a new folder in Packages, and either create a new file in the project’s root folder, .gitmodules, or modify said file if the project already has at least one submodule in it.

Commit these changes. Note: do not open Unity quite yet!

# Assuming we're already at the root folder of the project
git add .gitmodules Packages/[Folder]
git commit -m "Adding new submodule to Package folder."

4. Update package.json via text editor

Navigate to the new submodule folder, and open the file, package.json in your favorite text editor. Observe the first few lines…

{
  "name": "com.omiyagames.template",
  "displayName": "Omiya Games - Template Unity Package",

Change the information under the name and displayName to string values appropriate for your package. For name, the typical format is com.[Company Name].[Package Name]; displayName is what Unity uses to display the package, and can be named any short, single-line text. For example, with String Cryptographer, I changed the fields to:

{
  "name": "com.omiyagames.stringcyprtographer",
  "displayName": "Omiya Games - String Cryptographer",

It’s also recommended to edit the license, author, and keywords field to what’s appropriate for your package. For reference, here’s what the Template Unity Package has filled in:

"license": "MIT",
"author": {
  "name": "Taro Omiya",
  "email": "support@omiyagames.com",
  "url": "https://www.omiyagames.com"
},
"keywords": [
  "Omiya Games",
  "Scripts",
  "Library"
],

5. Update all *.asmdef files via text editor

Next, navigate into the Runtime folder, and look for the Assembly Definition file, OmiyaGames.Template.asmdef. With TortoiseGit, right-click on it, and select “TortoiseGit -> Rename…”

Rename the file appropriate to the package, following the Unity file naming convention: [CompanyName].[PackageName].asmdef. For example, with String Cryptographer, I changed the name to:

The command line equivalent is, of course, git mv:

# Assuming we're at the Runtime folder
git mv OmiyaGames.Template.asmdef [Company].[PackageName].asmdef

The command above helps Git file tracking, especially in comparison to the more common add and remove commands would.

Anyways, rinse and repeat renaming files for the following:

• Editor/OmiyaGames.Template.Editor.asmdef, with the naming convention [CompanyName].[PackageName].Editor.asmdef;
• Tests/Runtime/OmiyaGames.Template.Tests.asmdef, with [CompanyName].[PackageName].Tests.asmdef;
• And finally, Tests/Editor/OmiyaGames.Template.Editor.Tests.asmdef, with [CompanyName].[PackageName].Editor.Tests.asmdef.

For safety, I recommend committing all these changes. With TorttoiseGit, right-click on the new packager folder (which, remember, is now a nested Git repository), and select commit.

For command line, simply changing to the submodule folder, and running Git commands in it should be sufficient:

# Assuming we're already in the submodule folder (Packages/[Folder])
git add package.json
git commit -m "Updating package.json name fields, and renaming all the *.asmdef files appropriately."

6. Generate meta files with Unity

Now it’s time to open Unity! After a couple minutes of importing, the new package should appear under the project pane.

If you haven’t done this already, it is highly recommended to set the “Project Settings -> Editor -> Version Control Mode” Unity setting to Visible Meta Files, and “Project Settings -> Editor -> Asset Serialization Mode” to Force Text. This will enhance the project to be more version-control-friendly.

These settings will generate new *.meta files for each asset in the submodule. For those who don’t know, meta files contains import settings of their respective file, and more importantly, their unique GUID. Again for safety, it’s recommended to add and commit these files. We can later remove unnecessary sample files.

7. Update package.json with Unity

In Unity, under the Project window, search for package (the window won’t show the file extension), and click on it. Once selected, Unity will update the Inspector to provide a nice and easy-to-read interface to change most fields under the package.json file.

In particular, it is recommended to update the following fields:

• the package version:
  • The tooltip for this field recommends the [Release].[Feature].[BugFix](-preview.[Revision]) numbering format;
• package type,
• the minimum version of Unity this package supports,
• a brief description for the package.,
• and this package’s dependencies:
  • Note: each listed package name should follow the com.[company].[package-name] convention.
  • Version, of course, corresponds to the dependency’s version this package expects.

Like all Unity import settings, once you’ve finalized your changes, click the Apply button at the bottom of the Inspector dialog to update the JSON file.

8. Update all *.asmdef files with Unity

Under the Project window, again, select the Assembly Definition file in the Runtime folder. The Inspector should update to provide the information stored in that file.

Start by updating the Name field to match the filename (without the file extension) itself. If appropriate, go through the checkbox list, and update any options that are relevant to your project. For this package, I left all the checkboxes to their default state. Remember, again, to finalize your changes by clicking the Apply button at the bottom of the Inspector dialog to update the *.asmdef file.

Then select the Assembly Definition file in the Editor folder. Like the Runtime file, update the Name field, and any other appropriate checkboxes. Unlike Runtime file, uncheck the Use GUIDs checkbox under Assembly Definition References group, highlight the listed item, and click the minus button below the list to remove the placeholder element. Click on the plus button below the list to create a new (Missing Reference) entry, then drag-and-drop the Runtime Assembly Definition file into the field.

This helps establish that the Runtime C# project is a dependency to the Editor C# project. Remember to click on the Apply button after this.

Repeat the same process again for the Tests/Runtime asmdef file, which has the same dependency as Editor. Edit 4/29/2020: as of version 1.2.2, Template Unity Package added UnityEngine.TestRunner and UnityEditor.TestRunner as a dependency to Tests/Runtime asmdef file. As that dependency is necessary to run Nunit test, do not remove it from the list!

Finally, for the Tests/Editor asmdef file, add both the Runtime and Editor asmdef files to complete the dependency list. Edit 4/29/2020: like the Tests/Runtime above, make sure to leave behind the UnityEngine.TestRunner and UnityEditor.TestRunner dependencies in the list.

As usual, it’s recommended to add and commit these modifications.

8.1. (Optional) push changes to the submodule repository

It might be a good idea at this point to also push the changes in the submodule to the online repository. With TortoistGit, this can be done with Git Sync... dialog:

The dialog will display all the commit logs that has yet to be pushed to the repository. Confirm the changes, then click on the button, Push.

For command line, as always, the operation is as simple as running git push origin.

# Assuming we're already in the submodule folder (Packages/[Folder])
git push origin

9. Move the project files to the appropriate package folder

Now it’s time to develop our Package the way we want it! First, it might be a good idea to get rid of the following example C# scripts:

• Editor/EditorExample.cs
• Runtime/RuntimeExample.cs
• Tests/Editor/EditorExampleTest.cs
• Tests/Runtime/RuntimeExampleTest.cs

Next, it’s worth evaluating what files to put into the package. Basically, any files that goes in the package will not be editable by the user. Good candidates includes scripts and default assets. For files that is intended to be imported into one’s project — similar to the asset store — will be covered in the next step (step 10).

Once you’ve determined which files to move from the project into the package, it’s recommended to use the Unity Project window to drag-and-drop files into the package’s folder. This will move the meta file to the package. As for which folder to put it in:

• Most script and default asset files should go under the Runtime folder. If the file doesn’t fall under any of the categories below, drop it in here.
• If a script uses UnityEditor, or an asset only appears in the editor, e.g. Gizmos, those files will belong in the Editor folder.
• Unit test scripts on C# files under Runtime folder should, of course, go under the Tests/Runtime folder.
• Similarly, unit test scripts for Editor files should go under the Tests/Editor folder.

So long as the folder naming scheme above is preserved, one can add as many folders under each of the above folders as they like. Also for C# scripts, it’s recommended to wrap your scripts under a common namespace for your package.

Again, it’s recommended to add and commit these modifications in both the submodule and full Unity project.

9.1. (Optional) Troubleshooting dependencies

Sometimes, after moving a script into a package, Unity will provide an error indicating either a namespace or class isn’t found.

This can happen if the script relies on another package to be imported. The easiest way to resolve this is by updating the Assembly Definition file in the folder the script in question is in, and adding the missing references from other packages into the Assembly Definition References list. See step 8 on how to do this.

9.2. (Optional) Troubleshooting Editor namespace conflict

If like me, you’ve added namespaces to your code to match the package naming scheme, in the editor files, you’ve probably encountered an error message like this:

This is due to the result of naming the namespace to end with .Editor, thus creating a naming conflict with Unity’s Editor class. Fortunately, there’s an easy fix: simply rename the base class, Editor, to UnityEditor.Editor.

9.3. (Optional) Renaming elements

If you are using Visual Studio, the IDE Unity recommends installing with the game engine, there’s a convenient refactoring tool for renaming classes, variables, methods, and even namespaces in C# scripts. By simply right-clicking on a name, and selecting “Rename…” the name will highlight itself, giving you an opportunity to type in a new name.

Doing so, then hitting the enter key will not only update the script in question, but also rename that element in all the other scripts that references it, including the Unity project files. The operation is reversible with the usual undo command as well. Don’t forget to save all the edited files when this process finishes!

9.4. (Optional) Deleting unnecessary folders

If any of the Unity folders end up only containing a single asmdef, its meta file, and nothing else, it’s OK to remove said folder. For example, if you don’t plan on adding any unit tests to your packages, the Tests folder and all of its content can be deleted.

10. Move the sample assets to the appropriate package folder

As most Unity users know, it’s possible to import assets from the Package Manager via the Samples section and the “Import into Project” button.

To create sample assets that can be imported through this method, start by creating one or more folders under the Samples~ folder, using your favorite file browser (or command line). Each folder can be setup to be listed under the Package Manager as a unique set of assets to import into the project. Note that the Template Unity Package already has one named Example1; feel free to delete or continue using this folder.

Unlike package files, sample files are not expected to have a meta file attached with them. Therefore, it’s recommended to copy files from the larger project into the individual Samples~ folder using one’s favorite file browser (or, again, command line).

Once all the files are copied over, open the package.json file in your favorite text editor. Search for the samples field, which takes in a list of hashes. Copy the hash that already in the file, and edit the fields appropriately:

• displayName is a single-line string that Unity uses to display the name of the sample.
• description is a single-line description of the sample’s purpose.
• path is the location of the folder, using / as the folder divider. Path is relative to the root of the package folder, so most samples will follow the format, Samples~/[folder name].

For example, if there are two sample projects, one under Samples~/Example1, and the other, Samples~/Example2, your samples field may look like the following:

"samples": [
  {
    "displayName": "Example 1",
    "description": "This sample is just an example",
    "path": "Samples~/Example1"
  }, 
  {
    "displayName": "Example 2",
    "description": "This yet another example sample",
    "path": "Samples~/Example2"
  }
],

Obviously, if your project doesn’t require samples, simply removing the samples field from the package.json file, and deleting the Samples~ folder and its content should suffice.

Once again, it’s recommended to add and commit these modifications in both the submodule and full Unity project (if the latter has any changes from this step for whatever reason).

11. Update Markdown files

Once the package is in a stable state, it’s time to update the licensing and documentation files. The following text files should be updated with your favorite text editor, documented in Markdown format:

• README.md
  • Provides an introduction to the user using the package. More useful for Github, Bitbucket, and OpenUPM web interfaces, which automatically formats the file to something easier to read.
• LICENSE.md
  • Contains the package’s overall license (e.g. MIT License). If the package uses third-party files, then it’s recommended to indicate as such in a different file below:
• THIRD PARTY NOTICES.md
  • This optional license file lists third-party tools this package utilizes, and their licenses. Unity provides their own template on how to format this list.
• CHANGELOG.md
  • Contains a list of changes made between each package version. This file should be updated with new information each time a new version of the package is being prepared.

It might be a good idea to add and commit these modifications before updating the last set of documentation files below.

The last markdown file to update is the Documentation~/Template.md file. Except, per Unity’s recommendation, the file should be renamed to reflect the package name. As such, it’s recommended to use Git rename/move command to rename the file first before editing it. As the folder name implies, this file should contain the detailed documentation of the scripts and assets used in the package.

For efficiency, the Template Unity Package also provides a Doxyfile, which one can use to load template settings into Doxywizard, update said settings, and generate a set of web files from the source code alone.

With all the package files updated, add and commit any left-over modifications in the submodule. In the next step, we’ll be pushing them into the online repository.

12. Tag and push all changes

Now that package development is complete, it’s time to prepare for a first release. If you haven’t done so already, it’ll be a good idea to add and commit all the changes made to both the package submodule and the parent Unity project. This can be done by simply running the Git commands (both TortoiseGit and command line) in their respective folders. Furthermore, if you have been working off of another branch besides master in your package submodule, now would be a good time to merge all those changes into the master branch.

For the parent Unity project, it’ll be a good idea to push all the changes to the online repository as conveyed in Step 8.1. This will add the submodule folder into the remote repository, storing the commit hash the submodule should grab when the entire project is cloned later (with the git clone --recurse-submodules [url] command). Github, for example, will automatically link the submodule folder to the corresponding repository on their web interface, making the connection clear.

While Unity package manager currently doesn’t support the Git tag feature, for the package submodule, it’ll still be a good idea to mark the commit the first release was made in. Under the package submodule folder, in the master (i.e. default) branch, right-click and find the context-menu option for “TortoiseGit -> Create Tag…”

In the Create Tag dialog, assign a tag name, such as the same version number used in package.json, and provide a clear message on what the purpose of the tag is for. Click OK.

Finally, bring up the Git Sync menu again. Click on the triangle next to the Push button, and select “Push tags.”

For command line, it’s much more preferable to use annotated tags instead of lightweight ones, as it stores useful extra metadata such as a log. The operation to do this is pretty simple:

# Assuming we're on the submodule folder, master branch

# Flag '-a' indicates the tag is annotated.
# Flag '-m' indicates the string that follows
# is the message to store with the tag.
git tag -a 0.1.0-preview.1 -m "First release of cryptography package, which adds the tool, String Cryptographer."

# Push the submodule to the online repository.
# Flag '--tags' is necessary to push the tags as well.
git push origin --tags

This completes the creation of a new Unity package. The example I used can be found in this Github repository: github.com/OmiyaGames/omiya-games-cryptography

Verification

If the Git repository you’ve created for your package is open-source, verification is as easy as creating a new Unity package, and adding the git URL to the package manager.

Naturally, it’ll be easier for Unity to enter the HTTPS URL under the repository field.

Note that Unity does not resolve dependencies that are also Git repositories. If your repository has an external dependency like the example Cryptography package does, import the dependency packages first.

That’s basically everything there is to it!

(Optional) Adding package to OpenUPM

A fairly recent development is the OpenUPM project: a collection of open-source Unity packages, and an open-source command line tool to import said packages more easily. The command line tool in particular provides some extra features Unity’s own package manager doesn’t have, including dependency resolution and support of Git tags. Contributing to this awesome project is quite simple.

Note: as far as this writer can tell, OpenUPM requires packages to be hosted on Github.

1. Fork the OpenUPM repository

The OpenUPM project works by checking if there are any new YAML files added into its Github repository. Github, of course, requires you to propose patches to the repository for it to be uploaded through their web interface. While the OpenUPM website does provide an easy-to-read web form to create new files, it’ll still be a good idea to prepare your own fork of its repository beforehand, if you haven’t done so already.

Simply login to Github if you’re not already, navigate to github.com/openupm/openupm, and click on the Fork button.

2. Fill out the submission form

Open the OpenUPM submission page, and fill out the submission form.

Click on Verify package to let the website evaluate your repo’s package.json file. This should only take a few seconds. If successful, OpenUPM will provide a preview of what the new YAML file will look like.

3. Create a new YAML file

Click on Upload Package, and OpenUPM will navigate to Github with a new YAML file (which, format-wise, is quite similar to JSON), ready to be added into your forked repository. If there are any missing details or mistakes in it, make edits as necessary. Once finished, scroll to the bottom, then write up a message explaining the new addition.

Click Propose new file to create the file into your forked repository.\

Note: this writer had to make edits to the file (like add and delete some spaces) to re-enable the Propose new file button. Not sure if that’s a bug with Github.

4. Merge the file to OpenUPM

In the next Github window, click on Create pull request button.

This, of course, gives you an opportunity to add a log message explaining the request. Fill out as necessary, then click on the Create pull request button again, this time below the log message.

5. Verification

And that’s it! You’re all done! The rest will be handled by OpenUPM automatically. Some of its progress will be visible on the Github pull request created in the steps above. For this writer, this process took a couple of minutes.

You can verify the package has been successfully uploaded by checking openupm.com/packages/[name.of.package.here]. For our example, the URL was openupm.com/packages/com.omiyagames.cryptography. The package page should have a printout of your README.md file.

Again, do note that even if the Github pull request process has fully completed, the build system on the OpenUPM site may need some time to catch up to prepare the package webpage. If it’s still not up, wait an hour or two, and check again.

The instructions on how to install and use the OpenUPM command line tool is available here: openupm.com/docs/getting-started.html

Happy coding!

Categories: Game Development, Tutorial | Tagged: C#, Command Line, Game Development, Git, Github, Guide, Instructions, JSON, Markdown, OpenUPM, package, Template, Tools, TortoiseGit, Tutorial, Unity, Unity 3D