Duget Package Manager for Delphi

[Estimated Reading Time: 4 minutes]

In a few of my recent posts I’ve mentioned a thing called duget and dropped various hints as to what it is, or just coming right out and saying. It is (or will be) a package manager, similar to nuget which any .net developer will already be familiar with.

As promised, this post provides an overview of the current state of this project and is the first in a quick-fire series to explore the key features.

In simple terms, duget is at an early MVP stage. Here’s what it currently can do:

• Create packages
• Push packages to configured feeds
• Restore packages into a project
  • Identify and pull required packages
  • Unpack packaged source files
  • Update project files with search paths and/or rewriting project cfg files (for command-line compilation scenarios)
• Update project dependencies
  • Identify later versions of referenced packages and update dependency files with more current versions

So where’s 1.0.0 already !?! Well, there are a few limitations:

• THE BIG ONE: Currently only file system based feeds are supported (folders/file shares)
• Incomplete command line Api
• The process of updating project search paths only supports DPROJ project files. The Api is extensible to other project file formats, I just haven’t addressed DOF or BDSPROJ yet

Although the first limitation is called out as THE BIG ONE, this isn’t actually what’s holding up a public release. Even a strictly file-share based package manager is useful (I know, I’m using it!)

In addition, there are no command line facilities for working with the various configuration files. These have to be edited manually and in some cases they are still evolving. I don’t want to release a 1.0 specification for those until I feel things have settled down some more. Also it is important to note that duget is currently designed for packaging source files as this is my own primary use case. Some additional thought is needed for dealing with packaging binaries (dcus) and especially components (i.e. IDE packages).

However, as I have mentioned, I am already at a point where I can use duget in my own build pipelines and processes, though this still needs a bit of hand-holding from time to time.

There are a couple of other major milestones I wish to reach before an initial public release:

1. A stable channel for distributing the tool itself (ideally with a reliable auto-update mechanism or at least a notification)
2. Refactoring the build system to “dog-food” itself, with it’s own duget dependencies and all dependencies packaged for consumption
3. A comprehensive code-level Api (I’m hoping that someone with experience of creating IDE plug-ins could run with the job of creating a “Package Manager Console” UX for various Delphi IDE’s)

What does this all look like in current state ?

---

The PACK Command

It starts with a package specification file. This is a JSON file that describes the content of a package to be created by the packager. Here’s an example for the Smoketest 2.0 package:

{
    id: "deltics.smoketest",
    summary: "A lightweight and efficient unit testing framework for Delphi versions 7 thru 10.3 (excl. 8 and 2005) with support for custom results writers to emit test results in whatever format is required.  XUni2 2.x format is supported OOB.",
    tags: ["deltics", "delphi", "unit testing"],
    language: "pascal",
    compiler: "delphi",
    compiler-versions: "[7 .. 10.3], !8, !2005",
    platform: "windows",
    copyright: "\u00A92019 Jolyon Direnko-Smith",
    language: "en",
    content: "source",
    dependencies: [
    	"deltics.inc:1.0.1"
    ],
    source: {
        files: [    
            "src\\Deltics.Smoketest.inc",
            "src\\Deltics.Smoketest.pas",
            "src\\Deltics.Smoketest.ResultsWriter.pas",
            "src\\Deltics.Smoketest.ResultsWriter.XUnit2.pas",
            "src\\Deltics.Smoketest.Test.pas",
            "src\\Deltics.Smoketest.TestResult.pas",
            "src\\Deltics.Smoketest.TestRun.pas",
            "src\\Deltics.Smoketest.Utils.pas"
        ]
    }
}

You will note that this isn’t strictly speaking standards compliant JSON. The only deviation from the standard however is that object property names do not need to be quoted. This is a feature of my own JSON library which allows for this flexibility since I feel this makes JSON-as-config much more readable and livable with. My library can also handle 100% standards compliant JSON of course, so if you produce config files with some other JSON serialisation library, duget will still be happy.

A large number of the properties in this file are currently ignored. In fact, there are just three which are crucial:

• id
• dependencies
• source.

The id determines the id of this package (duh!) and is crucial to the identity of the package through its lifetime.

The dependencies property identifies other packages that this package itself takes a dependency on. In this case, only one other package is required – the deltics.inc package, specifically version 1.0.1 of that package.

The source property then identifies the files that are to be part of this package.

The filename in this case is simply .duspec since it is the only package specification in the folder. If a folder contained source for multiple packages then you could name them however you wish, to keep them separate, as long as they have a .duspec extension. The actual filename is not important – the package id comes from the spec content, not the spec filename.

Why No Version Property ?

You may have noticed that there is no property that identifies the version of the package itself. A version property is supported and if one is specified in the spec then this will be used to version the package. However, in a CI/CD scenario this would mean that a build that incorporated a step to create a package would ideally then ‘bump’ the version number and then need to commit that change back to the repo.

To avoid this, I envisage that the build will determine the version number and that this will be provided to the packaging command. The version number then does not need to be persisted in the specification and indeed should not be, to avoid potential confusion.

Whether a version is specified in the spec or not, any version provided on the command line will take precedence (a warning will be emitted if there is a version in the spec that is overridden by the command line). If there is no version in the spec or on the command line, then the package command will fail.

So, to create the package described by the above spec, you would use a command line similar to:

duget pack --version:2.0.2

This assumes you are running duget from the folder containing the .duspec file and that you wish to package all specs. You can package a different folder by providing a --path switch and you can package only a specific spec by providing that as an argument. In other words for a single package in a spec file named simply <code>.duspec in the current folder, the above command is exactly equivalent to:

duget pack .duspec --path:. --version:2.0.2

If successful, the pack command creates the package in the folder alongside the spec. The package file is named for the package id and version, with a .dupkg extension. In the above example, this will produce a package:

deltics.smoketest-2.0.2.dupkg

Just like nuget, a package is simple a zip file.

---

Tomorrow I’ll show how duget makes packages available for consumption by other projects.