{"id":2979,"date":"2019-09-24T18:07:42","date_gmt":"2019-09-24T06:07:42","guid":{"rendered":"https:\/\/www.deltics.co.nz\/blog\/?p=2979"},"modified":"2019-09-24T18:07:49","modified_gmt":"2019-09-24T06:07:49","slug":"duget-package-manager-for-delphi","status":"publish","type":"post","link":"https:\/\/www.deltics.co.nz\/blog\/posts\/2979\/","title":{"rendered":"Duget Package Manager for Delphi"},"content":{"rendered":"[Estimated Reading Time: <\/span> 4<\/span> minutes]<\/span><\/span>\n

In a few of my recent posts I’ve mentioned a thing called duget<\/strong> and dropped various hints as to what it is, or just coming right out and saying. It is (or will be) a package manager, similar to nuget<\/strong> which any .net developer will already be familiar with.<\/p>\n\n\n\n

As promised, this post provides an overview of the current state of this project and is the first in a quick-fire series to explore the key features.<\/p>\n\n\n\n\n\n\n\n

In simple terms, duget<\/strong> is at an early MVP stage. Here’s what it currently can do:<\/p>\n\n\n\n

  • Create packages<\/li>
  • Push packages to configured feeds<\/li>
  • Restore packages into a project
    • Identify and pull required packages<\/li>
    • Unpack packaged source files<\/li>
    • Update project files with search paths and\/or rewriting project cfg<\/strong> files (for command-line compilation scenarios)<\/li><\/ul><\/li>
    • Update project dependencies
      • Identify later versions of referenced packages and update dependency files with more current versions<\/li><\/ul><\/li><\/ul>\n\n\n\n

        So where’s 1.0.0 already !?! Well, there are a few limitations:<\/p>\n\n\n\n

        • THE BIG ONE:<\/strong> Currently only file system based feeds are supported (folders\/file shares)<\/li>
        • Incomplete command line Api<\/li>
        • The process of updating project search paths only supports DPROJ project files. The Api is extensible to other project file formats, I just haven’t addressed DOF or BDSPROJ yet<\/li><\/ul>\n\n\n\n

          Although the first limitation is called out as THE BIG ONE<\/em>, this isn’t actually what’s holding up a public release. Even a strictly file-share based package manager is useful (I know, I’m using it!)<\/p>\n\n\n\n

          In addition, there are no command line facilities for working with the various configuration files. These have to be edited manually and in some cases they are still evolving. I don’t want to release a 1.0 specification for those until I feel things have settled down some more. Also it is important to note that duget<\/strong> is currently designed for packaging source<\/em> files as this is my own primary use case. Some additional thought is needed for dealing with packaging binaries (dcus) and especially components (i.e. IDE packages).<\/p>\n\n\n\n

          However, as I have mentioned, I am already at a point where I can use duget<\/strong> in my own build pipelines and processes, though this still needs a bit of hand-holding from time to time.<\/p>\n\n\n\n

          There are a couple of other major milestones I wish to reach before an initial public release:<\/p>\n\n\n\n

          1. A stable channel for distributing the tool itself (ideally with a reliable auto-update mechanism or at least a notification) <\/li>
          2. Refactoring the build system to “dog-food” itself, with it’s own duget<\/strong> dependencies and all dependencies packaged for consumption<\/li>
          3. A comprehensive code-level Api (I’m hoping that someone with experience of creating IDE plug-ins could run with the job of creating a “Package Manager Console<\/em>” UX for various Delphi IDE’s)<\/li><\/ol>\n\n\n\n

            What does this all look like in current state ?<\/p>\n\n\n\n

            \n\n\n\n

            The PACK Command<\/h2>\n\n\n\n

            It starts with a package specification file. This is a JSON file that describes the content of a package to be created by the packager. Here’s an example for the Smoketest 2.0<\/strong> package:<\/p>\n\n\n\n

            {\n    id: "deltics.smoketest",\n    summary: "A lightweight and efficient unit testing framework for Delphi versions 7 thru 10.3 (excl. 8 and 2005) with support for custom results writers to emit test results in whatever format is required.  XUni2 2.x format is supported OOB.",\n    tags: ["deltics", "delphi", "unit testing"],\n    language: "pascal",\n    compiler: "delphi",\n    compiler-versions: "[7 .. 10.3], !8, !2005",\n    platform: "windows",\n    copyright: "\\u00A92019 Jolyon Direnko-Smith",\n    language: "en",\n    content: "source",\n    dependencies: [\n    \t"deltics.inc:1.0.1"\n    ],\n    source: {\n        files: [    \n            "src\\\\Deltics.Smoketest.inc",\n            "src\\\\Deltics.Smoketest.pas",\n            "src\\\\Deltics.Smoketest.ResultsWriter.pas",\n            "src\\\\Deltics.Smoketest.ResultsWriter.XUnit2.pas",\n            "src\\\\Deltics.Smoketest.Test.pas",\n            "src\\\\Deltics.Smoketest.TestResult.pas",\n            "src\\\\Deltics.Smoketest.TestRun.pas",\n            "src\\\\Deltics.Smoketest.Utils.pas"\n        ]\n    }\n}<\/code><\/pre>\n\n\n\n
            You will note that this isn’t strictly speaking standards compliant JSON.  The only deviation from the standard however is that object property names do not need to be quoted.  This is a feature of my own JSON library which allows for this flexibility since I feel this makes JSON-as-config much more readable and livable with.  My library can also handle 100% standards compliant JSON of course, so if you produce config files with some other JSON serialisation library, duget<\/strong> will still be happy.<\/p>\n\n\n\n
            A large number of the properties in this file are currently ignored.  In fact, there are just three which are crucial:<\/p>\n\n\n\n
            • id<\/code><\/li>
            • dependencies<\/code><\/li>
            • source<\/code>.<\/li><\/ul>\n\n\n\n
              The id<\/strong> determines the id of this package (duh!) and is crucial to the identity of the package through its lifetime.<\/p>\n\n\n\n
              The dependencies<\/strong> property identifies other packages that this package itself takes a dependency on.  In this case, only one other package is required – the deltics.inc<\/code><\/strong> package, specifically version 1.0.1<\/code> of that package.<\/p>\n\n\n\n
              The source<\/strong> property then identifies the files that are to be part of this package.<\/p>\n\n\n\n
              The filename in this case is simply .duspec<\/code> since it is the only package specification in the folder.  If a folder contained source for multiple packages then you could name them however you wish, to keep them separate, as long as they have a .duspec<\/code> extension.  The actual filename is not important – the package id<\/strong> comes from the spec content, not the spec filename.<\/p>\n\n\n\n
              Why No Version Property ?<\/h4>\n\n\n\n
              You may have noticed that there is no property that identifies the version<\/strong> of the package itself.  A version property is<\/em> supported and if one is specified in the spec then this will<\/em> be used to version the package.  However, in a CI\/CD scenario this would mean that a build that incorporated a step to create a package would ideally then ‘bump’ the version number and then need to commit that change back to the repo.<\/p>\n\n\n\n
              To avoid this, I envisage that the build will determine the version number and that this will be provided to the packaging command.  The version number then does not need to be persisted in the specification and indeed should not be, to avoid potential confusion.<\/p>\n\n\n\n
              Whether a version is specified in the spec or not, any version provided on the command line will take precedence (a warning will be emitted if there is a version in the spec that is overridden by the command line).  If there is no version in the spec or on the command line, then the package command will fail.<\/p>\n\n\n\n
              So, to create the package described by the above spec, you would use a command line similar to:<\/p>\n\n\n\n
              duget pack --version:2.0.2<\/code><\/p>\n\n\n\n
              This assumes you are running duget<\/strong> from the folder containing the .duspec<\/code> file and that you wish to package all<\/em> specs.  You can package a different folder by providing a --path<\/code> switch and you can package only a specific spec by providing that as an argument.  In other words for a single package in a spec file named simply <code>.duspec<\/code><\/code><\/strong> in the current folder, the above command is exactly equivalent to:<\/p>\n\n\n\n
              duget pack .duspec --path:. --version:2.0.2<\/code> <\/p>\n\n\n\n
              If successful, the pack command creates the package in the folder alongside the spec.  The package file is named for the package id and version, with a .dupkg extension.  In the above example, this will produce a package:<\/p>\n\n\n\n
              deltics.smoketest-2.0.2.dupkg<\/code><\/p>\n\n\n\n
              Just like nuget<\/strong>, a package is simple a zip<\/em> file.<\/p>\n\n\n\n
              \n\n\n\n
              Tomorrow I’ll show how duget makes packages available for consumption by other projects.<\/p>\n","protected":false},"excerpt":{"rendered":"
              [Estimated Reading Time: <\/span> 4<\/span> minutes]<\/span><\/span> An overview of the the duget package manager, outlining the current state of this project and the first in a quick-fire series to explore the key features.<\/p>\n","protected":false},"author":2,"featured_media":2981,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"Introducing Duget: A Package Manager for Delphi","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[324,323,4,321,346],"tags":[292,329,344,345],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p1TKYv-M3","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2983,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2983\/","url_meta":{"origin":2979,"position":0},"title":"Duget – PUSH-ing Packages","date":"25 Sep 2019","format":false,"excerpt":"We now know a little more about this duget thing, and have seen how to create a package. But a package cannot be consumed 'in situ' - it must be made available via a feed. Which brings us to the PUSH command.NOTE: Don\u2019t worry, I have my priorities straight. This\u2026","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2985,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2985\/","url_meta":{"origin":2979,"position":1},"title":"Duget – RESTORE, UPDATE and What’s Next ?","date":"26 Sep 2019","format":false,"excerpt":"Wrapping up this short series on duget, discussing RESTORE and UPDATE, arguably most important and useful commands in the duget repertoire.","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2936,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2936\/","url_meta":{"origin":2979,"position":2},"title":"Azure DevOps – Iterative Insertion Fixed!","date":"19 Sep 2019","format":false,"excerpt":"I figured out the iterative insertion problem and my build pipeline is now TIGHT! Fixing it was super-easy in fact, barely an inconvenience.","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/pipelines-hero-code-1024x256.jpg?fit=1024%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3000,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/3000\/","url_meta":{"origin":2979,"position":3},"title":"VersionInfo and Semantic Versioning","date":"08 Oct 2019","format":false,"excerpt":"How to make Semantic Version make sense in a VERSIONINFO world without Semantic Versioning.","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/bart-simpson-generator-1.png?fit=1024%2C343&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2919,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2919\/","url_meta":{"origin":2979,"position":4},"title":"Azure DevOps – Now You Too Can Use My Template(s)","date":"14 Sep 2019","format":false,"excerpt":"Learn how you too can use my Delphi build template in your own Azure DevOps pipelines, and a sneak preview of something special coming soon...","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/github-512.png?fit=512%2C512&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2931,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2931\/","url_meta":{"origin":2979,"position":5},"title":"‘Sorry, this script is too long…’","date":"18 Sep 2019","format":false,"excerpt":"An object lesson on the importance of defensive programming and providing helpful error messages to your users.","rel":"","context":"In "Azure DevOps"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/Mark_Twain_life_1900s.jpg?fit=883%2C331&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2979"}],"collection":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/comments?post=2979"}],"version-history":[{"count":4,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2979\/revisions"}],"predecessor-version":[{"id":2991,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2979\/revisions\/2991"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media\/2981"}],"wp:attachment":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media?parent=2979"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/categories?post=2979"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/tags?post=2979"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}