{"id":2985,"date":"2019-09-26T18:57:57","date_gmt":"2019-09-26T06:57:57","guid":{"rendered":"https:\/\/www.deltics.co.nz\/blog\/?p=2985"},"modified":"2019-09-26T18:58:05","modified_gmt":"2019-09-26T06:58:05","slug":"duget-restore-update-and-whats-next","status":"publish","type":"post","link":"https:\/\/www.deltics.co.nz\/blog\/posts\/2985\/","title":{"rendered":"Duget – RESTORE, UPDATE and What’s Next ?"},"content":{"rendered":"[Estimated Reading Time: <\/span> 6<\/span> minutes]<\/span><\/span>\n

So far we’ve look at creating<\/a> and publishing<\/a> packages using duget<\/strong>. This is of course an essential part of the process, but the real power comes when using those packages in your projects and ensuring that any dependencies you take also bring in their required dependencies and so on.<\/p>\n\n\n\n

This last post in this short series shows how that currently works in duget<\/strong>.

As with the previous post<\/a><\/em>, <\/em><\/strong>this was written before Liev arrived<\/a>. \ud83d\ude42<\/em><\/p>\n\n\n\n\n\n\n\n

Which brings us to the final file of interest: the project configuration file. This identifies the packages that a particular project takes dependencies on. Every project has potentially two such files that determine the dependencies for that project. The first is the path<\/em> configuration. This is a .duget<\/code> file in the same folder as the project. The second is a <<em>project<\/em>>.duget<\/code> file, sharing the same filename as the dpr<\/strong> but with the .duget<\/code> extension.<\/p>\n\n\n\n

In complex projects with multiple dprs<\/strong><\/em> involved having common dependencies, this allows those dependencies to be specified just once at the path level, with only project specific<\/em> dependencies then added if\/as needed in individual project files. If no project configuration is provided then the path configuration applies and vice versa.<\/p>\n\n\n\n

Project and Path Configuration files provide facilities to identify dependencies that are specific to particular build configurations or platforms. That is, your debug build configuration may take additional dependencies, e.g. on debugging tools\/frameworks, that your release builds do not.<\/p>\n\n\n\n

Project <\/strong>Group<\/strong><\/em> configurations are specifically not<\/em><\/strong> supported. This is because of the complication that a single project may conceivably exist in multiple<\/em> project groups leading to the possibility that the same project could have different dependencies depending on the project group context<\/em>. This simply does not make sense for project dependencies. <\/p>\n\n\n\n

Build Config and Platform specific dependencies are already supported by duget<\/strong> but is the area perhaps most likely to change as part of the evolution of the configuration files so I won’t go into the current implementation. Instead I’ll look at a simple scenario where a project simply takes dependencies that apply to all platforms and all build configurations:<\/p>\n\n\n\n

{\n  references:[\n    "&..\\\\.duspec",\n    "deltics.smoketest:2.0.2"\n  ]\n}<\/code><\/pre>\n\n\n\n
This example is taken from the configuration for the tests project of another library that I am currently packaging.  This project implements the tests that are run to verify that a library may be packaged, only if it passes these tests.<\/p>\n\n\n\n
For this reason, and intended only for use in such scenarios, this configuration includes a dependency on a package specification<\/em>.  This is denoted by the &<\/code> prefix on the reference, indicating that what follows is a path to a .duspec<\/code> file, rather than a package.  This tells duget<\/strong> that this project (the test project) directly<\/em> references the source for that package, as identified in the package specification<\/em>.<\/p>\n\n\n\n
The second dependency is more straightforward.  Since the tests project uses Smoketest 2.0<\/strong> to implement the tests, it takes a dependency on that required package.<\/p>\n\n\n\n
The RESTORE Command<\/h2>\n\n\n\n
To make sure that dependencies are available to a project for compilation, you simply use the restore<\/strong> command.  Again, the simplest version of this command will look for all .duget<\/code> files in the current folder:<\/p>\n\n\n\n
duget restore<\/code><\/p>\n\n\n\n
It actually looks for dpr<\/em><\/strong> files and then<\/em> for applicable<\/em> .duget<\/strong> files at the path and project configuration level to be applied to each dpr<\/em><\/strong>.<\/p>\n\n\n\n
For each project duget restore<\/strong> will “walk the dependency tree”, loading packages from the cache where possible, fetching them from feeds where needed and then repeating that process for any dependencies that those packages bring in.  It will also bring in any package dependencies<\/em> identified in the package specification for any source references (just referencing the source itself isn’t enough).<\/p>\n\n\n\n
If any dependency cannot be satisfied from the cache or any feed then the restore process fails.<\/p>\n\n\n\n
Just having the required packages in the cache isn’t enough of course.  As well as that, the restore<\/strong> command will unpack the source files from each package into a specially created folder under the projects.  Each package is unpacked into a folder named for the package id and version of the package involved.  So for two projects in the same folder referencing the same dependency, they share the unpacked folder.  e.g. the unpacked folder for deltics.smoketest:2.0.2 would be:<\/p>\n\n\n\n
<projectfolder><\/em>\n   \\__duget\n       \\deltics.smoketest-2.0.2<\/pre>\n\n\n\n
If  two separate projects in the same folder reference different versions of the same package<\/em>, then both will be unpacked but each project will reference only it’s required version. e.g. for two projects referencing different versions of deltics.smoketest<\/strong>:<\/p>\n\n\n\n
<projectfolder><\/em>\n   \\__duget\n       \\deltics.smoketest-2.0.1\n       \\deltics.smoketest-2.0.2<\/pre>\n\n\n\n
But the restore<\/strong> command doesn’t stop there.  It also updates the -I, -R and -U settings of any <project><\/em>.cfg<\/strong> file it finds with the paths to the required package folders in the __duget<\/code> folder, for that project.<\/p>\n\n\n\n
It also updates the search path in any dproj<\/strong> file.<\/p>\n\n\n\n
The search path added to either cfg<\/strong> or dproj<\/strong> files also includes the path to the source files for any source referenced package.  Source file references (those & prefixed .duspec references) are not<\/em> copied to the __duget<\/strong> folder but have their actual source locations added to the search path(s).<\/p>\n\n\n\n
Any non-duget<\/strong> paths present in the cfg<\/strong> or dproj<\/strong> settings already are preserved as-is.<\/p>\n\n\n\n
\n\n\n\n
The UPDATE Command<\/h2>\n\n\n\n
The last command to mention for now is duget update<\/strong>.  Since the restore<\/strong> command does not<\/em> update package versions, the update<\/strong> command takes care of this.  Separating the two is important as you would want a duget restore<\/strong> in a build process to reliably restore the same packages used for all previous builds and the same versions on the dev machines (unless and until someone commits a change to update those dependencies).<\/p>\n\n\n\n
duget update<\/strong> is a mechanism to simplify that business of updating dependencies on dev machines.  You would never run update<\/strong> on a build machine.<\/p>\n\n\n\n
duget update<\/strong> also provides an additional level of forgiveness in your dependency configurations.  If you don’t know the current version of a package that you wish to use, but you do know the package id, then you can add a reference to just that package id and duget update<\/strong> will identify the latest version for you.  restore<\/strong> requires that a version is identified (except for & source references, obviously).<\/p>\n\n\n\n
Again, the simplest possible invocation of the update<\/strong> command is:<\/p>\n\n\n\n
duget update<\/code><\/p>\n\n\n\n
And as with other commands, if you only wish to update a specific project (or projects) then specific projects may be identified as arguments.  Similarly if you only wish to update the version of a specific package (or packages) then this can be specified in a \u2014package<\/code> option, e.g.:<\/p>\n\n\n\n
duget update myproject \u2014package:some.package;other.package<\/code><\/p>\n\n\n\n
So if you’re starting a new test project and want to use the latest version of smoketest<\/strong> and\/or other packages but don’t know what the latest versions are, you could create a dependency file similar to:<\/p>\n\n\n\n
{\n  references:[\n    "deltics.smoketest",\n    \u201cfastmm\u201d\n  ]\n}<\/code><\/pre>\n\n\n\n
Run duget update<\/strong> over that, and watch as the dependencies are updated to the latest versions and your project automatically configured to use them!  You could also specify a version of course, and this may be useful if you specifically don\u2019t want the absolute latest but do want the latest compatible<\/em> version of some earlier major version.<\/p>\n\n\n\n
The only problem arises if a version is specified that doesn’t exist and there are not any versions later than that one available, in which case you\u2019re left with an invalid version in the dependency file.  So if you don’t know, you’re better off leaving it out.<\/p>\n\n\n\n
By default duget update<\/strong> will only update to a later, compatible release version of a package.  If you are willing to accept pre-release packages or breaking changes then as long as you have a feed configured which provides such releases, you can force duget update<\/strong> to take these with the addition of options on the command line:<\/p>\n\n\n\n
duget update --acceptPreRelease\nduget update --acceptBreaking\nduget update --acceptPreRelease --acceptBreaking<\/pre>\n\n\n\n
Whatever your appetite for change when updating, make sure you include a duget restore<\/strong> in your automated builds and you are good to go!<\/p>\n\n\n\n
\n\n\n\n
What Next<\/h2>\n\n\n\n
The duget<\/strong> system is being exercised extensively in my current efforts to package my own code (and itself) for consumption using it but I will soon be looking for beta testers.  If anyone would like to help with IDE tooling I’d be particularly interested to hear from you.<\/p>\n\n\n\n
I am also starting to look at extending the coverage of this tool to FPC<\/strong> and Elements<\/strong> compiler technologies.<\/p>\n\n\n\n
Developers using Elements<\/strong> for Android\/Java<\/em> or .net<\/em> projects of course have the vast wealth of packages served up by the existing package ecosystems for those platforms, but Elements<\/strong> also supports native code development and a package management solution could have a place there.<\/p>\n\n\n\n
If you’re interested to get involved either as a tester or even as a contributor, let me know in the comments.<\/p>\n","protected":false},"excerpt":{"rendered":"
[Estimated Reading Time: <\/span> 6<\/span> minutes]<\/span><\/span> Wrapping up this short series on duget, discussing RESTORE and UPDATE, arguably most important and useful commands in the duget repertoire.<\/p>\n","protected":false},"author":2,"featured_media":2981,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"Duget - RESTORE, UPDATE and What's Next ?  A look at consuming packages using the duget package manager.","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[324,323,4,321,346],"tags":[292,344,345],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p1TKYv-M9","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2979,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2979\/","url_meta":{"origin":2985,"position":0},"title":"Duget Package Manager for Delphi","date":"24 Sep 2019","format":false,"excerpt":"An overview of the the duget package manager, outlining the current state of this project and the first in a quick-fire series to explore the key features.","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2983,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2983\/","url_meta":{"origin":2985,"position":1},"title":"Duget – PUSH-ing Packages","date":"25 Sep 2019","format":false,"excerpt":"We now know a little more about this duget thing, and have seen how to create a package. But a package cannot be consumed 'in situ' - it must be made available via a feed. Which brings us to the PUSH command.NOTE: Don\u2019t worry, I have my priorities straight. This\u2026","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3000,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/3000\/","url_meta":{"origin":2985,"position":2},"title":"VersionInfo and Semantic Versioning","date":"08 Oct 2019","format":false,"excerpt":"How to make Semantic Version make sense in a VERSIONINFO world without Semantic Versioning.","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/bart-simpson-generator-1.png?fit=1024%2C343&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2919,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2919\/","url_meta":{"origin":2985,"position":3},"title":"Azure DevOps – Now You Too Can Use My Template(s)","date":"14 Sep 2019","format":false,"excerpt":"Learn how you too can use my Delphi build template in your own Azure DevOps pipelines, and a sneak preview of something special coming soon...","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/github-512.png?fit=512%2C512&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2936,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2936\/","url_meta":{"origin":2985,"position":4},"title":"Azure DevOps – Iterative Insertion Fixed!","date":"19 Sep 2019","format":false,"excerpt":"I figured out the iterative insertion problem and my build pipeline is now TIGHT! Fixing it was super-easy in fact, barely an inconvenience.","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/pipelines-hero-code-1024x256.jpg?fit=1024%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2931,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2931\/","url_meta":{"origin":2985,"position":5},"title":"‘Sorry, this script is too long…’","date":"18 Sep 2019","format":false,"excerpt":"An object lesson on the importance of defensive programming and providing helpful error messages to your users.","rel":"","context":"In "Azure DevOps"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/Mark_Twain_life_1900s.jpg?fit=883%2C331&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2985"}],"collection":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/comments?post=2985"}],"version-history":[{"count":6,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2985\/revisions"}],"predecessor-version":[{"id":2998,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2985\/revisions\/2998"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media\/2981"}],"wp:attachment":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media?parent=2985"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/categories?post=2985"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/tags?post=2985"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The restore<\/strong> command will not<\/strong> attempt to update<\/em> the version of any package to any more current version.  Whatever version is specified is the version that will be provided.  If multiple versions of a given package are referenced in the dependency tree for a given project then the latest compatible<\/em> version will be used.  If multiple incompatible<\/em> versions are referenced then the restore command fails.  \u201cCompatibility\u201d of versions is determined using Semantic Versioning<\/a> rules.<\/p>\n\n\n\n