Free Pascal compiler now supports dynamic packages

I am pleased to announce the first release of full dynamic package support for the Free Pascal Compiler and the Lazarus IDE. This feature has been a long time coming for the ecosystem, but a complete implementation is now available for download from this website.

Dynamic packages allow users to create and install components, controls, and Lazarus extensions in seconds without rebuilding the entire IDE. They can also be used in your own programs to develop and deploy plugins that expose native Pascal classes and types through dynamic package libraries. To support this functionality, I have made numerous improvements to the compiler, its runtime, and the Lazarus IDE. I have also created additional IDE extensions to improve the experience of working with Lazarus packages.

Dynamic packages are implemented as standard shared library files for Linux, Windows, and macOS. They contain additional metadata generated by the compiler that can be inspected and queried using new RTL units. The runtime also includes mechanisms for safely preserving thread variables and managing dynamic types, such as strings and dynamic arrays, between programs and packages or between multiple packages.

When deploying your program, you can still bundle everything into a single executable. Alternatively, you can divide it into multiple dynamic packages that can be replaced while your program is running, providing a plugin system.

Restrictions

Dynamic packages come with a few restrictions. First, a unit contained in a package cannot have the same name as the package itself. I have enhanced the default package system so that the generated Pascal source file automatically appends the word Package to the package name. For example, if you create a package called

ElectroTools.lpk

, the source file listing all the units contained in the package will be named

ElectroToolsPackage.pas

instead of

ElectroTools.pas

.

Another restriction is that unit names must be unique across packages. If someone else has written a unit named Widgets, you cannot install another package containing a unit with the same name. You must either choose which package to install or rename one of the conflicting units.

Dynamic packages are also larger than ordinary shared libraries because they contain a map of everything inside them. Stripping a dynamic package removes the metadata required for it to function as a dynamic package.

RTL Enhancements

type
  TPackageType = record
    { The unit that declares the type }
    UnitName: string;
    { The name of the type }
    Name: string;
    { The kind of the type }
    Kind: TTypeKind;
    { The RTTI of the type }
    Info: PTypeInfo;
    { The class if Kind is tkClass, otherwise nil }
    ClassType: TClass;
  end;

The runtime library has been enhanced to support dynamic packages, including the metadata required to inspect their contents and the mechanisms needed to safely manage dynamic types and thread variables across package boundaries.

A new unit

DynPkgTypes.pas

has been added to the RTL to help you with reading information from dynamic packages. The same tried and true RTTI information can be gather from a package to work with its contents. There are enough functions in the RTL to work with dynamic packages to do everything you need, including loading, enumerating, searching, and unloading packages at runtime.

Lazarus Enhancements

I have made several enhancements to the Lazarus IDE to take better advantage of dynamic packages.

The first thing you will notice when you open Lazarus is a new progress window that can appear outside the IDE. When you modify source code belonging to a package registered with the IDE, the system detects the changes and automatically rebuilds the package before Lazarus starts. While this is happening, a progress window notifies you that changes have been detected and the package is being rebuilt.

The same window appears when you choose to rebuild the entire IDE. It remains visible and displays information about the current operation, including the estimated time remaining before the IDE restarts. The progress window also appears when adding or removing packages if the operation takes more than five seconds. This can happen when installing multiple packages at once, which is another new feature.

Another enhancement is a new Packages window, docked on the right by default. It displays the packages you have recently opened, and the list persists between Lazarus sessions. You can browse the files within a package and double click an item to open it.

When you open an installed package and click Compile in its package window, any associated controls, components, or extensions are updated within a second or two. This significantly speeds up development because changes that previously required rebuilding the entire IDE can now be applied almost immediately.

The package menu, which lists available packages, has also been improved. Adding or removing packages now updates the IDE without requiring a rebuild.

Package options have been extended as well. Under IDE Integrations, you can now restrict a package to a specific platform or widgetset. If a package does not meet the current platform or widgetset requirements, it will not appear in the package list. The IDE will also ignore the package when it detects changes or would otherwise attempt to open it.

For example, suppose you develop a TWinControl implementation for the Windows IShellView COM interface. You can restrict the package so that it is available only when Lazarus is running on Windows.

Another enhancement I created to demonstrate the capabilities of dynamic packages is a new form or frame conversion feature for Lazarus extensions. It converts any form or frame into an installable control that is automatically added to the component palette. All controls, events, methods, and properties are copied to the new control.

This makes it possible to visually design and lay out new controls or easily reuse existing forms and frames in other projects.

Note: If you attempt to build a package that is currently being used by an open form, the IDE will ask you to save or close the form before building the package.

These features and more are available for download now at the location linked below. The complete changes to FPC and Lazarus have been committed to GitLab in my forked branch named dynamic packages.

If you encounter any problems, please report them through the GitLab issue tracker associated with the branches linked above. If you would like to discuss dynamic packages or ask questions, you can use the forum linked below. Please note that a moderator must approve your account before you can create new posts.

Helping Out

If you would like to help with this project, the site needs moderators. You can send me (sysrpl) a private message if you are interested in contributing.

I am also having difficulty working on macOS and producing the necessary builds. If anyone has an M1 Macintosh they would be willing to donate, please send me a private message. I would really like to work on the compiler code needed to support dynamic packages on the M1 architecture.