How best to add Plugin Capability to a Delphi program

I am looking to add the capability for users to write plugins to the program I have developed in Delphi. The program is a single executable with no DLLs used.

This would allow the user community to write extensions to my program to access the internal data and add capabilities that they may find useful.

I've seen the post at: http://stackoverflow.com/questions/8140/adding-plugin-capability but its answers don't seem transferrable to a Delphi program.

I would like, if possible, to add this capability and keep my application as a single executable without any DLLs or additional modules required.

Do you know of any resources, components or articles that would suggest how to best do this in Delphi, or do you have your own recommendation?

From stackoverflow

• You could have a look at Hydra from Remobjects. Not only will this allow you to add plugins, but also to mix win32 and .net.

• If plugins will be developed in Delphi or C++ builder, use packages + interfaces. Delphi OTA is a good example for that. If plugins will be language independent, COM is a good way to go.

  Addition: If you won't use COM, u may need to provide SDKs for each language. And datatype handling between different languages can be pain(for example delphi string type). Delphi COM support is excellent, u don't need to worry about kind of details. Those are mostly impilicit with Delphi COM support. Don't try to invent the wheel again. I'm surprised why people doesn't tend to mention about it.

  lkessler : I am not familiar with what the Delphi OTA is, and an initial Google search didn't help. Do you have a good link to a resource describing it?

  Rob Kennedy : Delphi OTA: Open Tools API. First Google hit: http://delphi.about.com/od/objectpascalide/a/wizardsexperts.htm

  AhmetC : OTA = Open tools api. It provides bunch of interfaces to use IDE's functions in an expert or desingtime packages. For example GExpert uses lots of OTA functions. OTA is not kind of generic plugin system for delphi programmers. It is spesific to delphi IDE. I thought it can be a good example for you.

  lkessler : Unfortunate URL: wizard-sex-perts.htm

• Actually, the accepted answer to the question you cite is quite appropriate for Delphi as well. Your plug-ins will be DLLs, and you can dictate that they should export a function with a certain name and signature. Then, your program will load the DLL (with LoadLibrary) and get the address of the function (with GetProcAddress). If the DLL doesn't load, or the function isn't there, then the DLL is not a plug-in for your application.

  After you have the address for the DLL, you can call it. You can pass the function an interface to something that represents the parts of your application that you wish to expose. You can also require that the function return an interface with methods that your application will call at various times.

  When testing your plug-in system, it will be wise to write a plug-in yourself with a language other than Delphi. That way, you can be more confident that you haven't inadvertently required everyone to use Delphi.

  mghie : +1 on the hint that plug-ins should be writeable in other languages than Delphi too. Otherwise it's far simpler to just include RemObjects Pascal scripting into your application and be done with it. Creating an interface to your internal objects is necessary in both cases, ...

  mghie : and with the internal scripting you don't have to invest any more work for the plug-in interface itself. Also, make sure that plug-ins can be tested, added and removed without the need to close your application.

  AhmetC : "You can pass the function an interface to..." Do u think other languages can use or implement delphi interfaces without COM ?

  Mason Wheeler : Mghie: PascalScript is great, but it's no replacement for a real plugin system because it only implements about 2/3 of the language feature set, and whenever someone asks Carlo Kok about the remaining features, he almost invariably says he has no plans to implement them.

  Rob Kennedy : Ahmet, Delphi interface _are_ COM interfaces. The language doesn't need particularly special support, though. Plain old C can do it just fine.

  mghie : @Mason Wheeler: Most probably he doesn't want to improve PascalScript (being a rival to payed-for RemObjects stuff) too much any more. In that light going to a plug-in system seems indeed best.

• I'm using plug-ins to implement most of the functionality in the game engine I'm building. The main EXE is made up of a script engine, a plug-in manager, some basic graphics routines and not much else. I'm using TJvPluginManager from the JEDI VCL library. It's a very good manager, and it can add just about anything you want to your program. Check out the demos included with the package to see how it works. The only downside is that it adds a lot of JCL/JVCL code to your program, but that's not really an issue if you're already using other JVCL components.

  mghie : Does TJvPluginManager limit the plug-ins to Delphi-written ones? How is the interface implemented anyway, via COM?

  Mason Wheeler : Yes, the plugins have to be written in Delphi. (Or maybe C++ Builder. Not sure.) The plugins have to be descended from the TJvPlugin base class, which is basically just a wrapper around your plugin functionality to let it talk to the manager. And you can implement it as a DLL or a BPL.

• I tried to make an overview of all such options some time ago. Together with my readers/commenters we built this list:

  • DLL/BPL loaded from the program.
  • DLL/BPL loaded from a sandbox (which could be another copy of the program or a specialized "server" application and which communicates with the main program via messages/sockets/msmq/named pipes/mailslots/memory mapped files).
  • COM (in any of its variants).
  • DDE (please not).
  • External program that communicates via stanard input/output.
  • External program that communicates via files (file name is a parameter to the program).
  • External program that works via drop folder (useful for batch processing).
  • External program that communicates with windows messages/windows sockets/msmq/named pipes/mailslots/memory mapped files/database publish-subscribe.

• The first question I would ask is, do you need the plugins to access the UI of your host application, or add any UI elements of their own? Or will the plugins be limited to querying and/or supplying data to your host app?

  The latter is much easier and opens up two possibilities. Others have already mentioned DLLs, which is the first way to go. Certain caveats apply - in general you should be interfacing with a dll using only the data types that are used in Windows API. THat way you can be sure that the plugin DLLs will understand your data types, no matter what language/compiler they were created in. (So for example, use PChars, not strings. Do not as a rule pass Delphi classes such as TStream to a DLL. This will appear to work in some cases, but is unsafe in general, because even if the DLL was compiled in Delphi, it may have been a different version of the compiler, with a slightly different idea of what TStream is). Google for using DLLs in Delphi, and you'll find plenty more tips.

  Another way that hasn't been mentioned yet is to enable scripting in your application itself. There are several very capable 3rd-party scripting engines, both commercial and free, and most of them allow you to exchange Delphi objects with the script. Some of these libraries support only Pascal as the scripting language, others will let you use Basic (perhaps better for beginner users) or other languages. See for example RemObjects Pascal Script (free) at http://www.remobjects.com/free.aspx .

  My favorite scripting solution at the moment is Python for Delphi (P4D, also free) from http://mmm-experts.com/Products.aspx?ProductID=3 It can interface beautifully with your classes via RTTI, and allows you to execute Python code in your Delphi app, as well as use Delphi classes in Python scripts. Given the popularity of Python, this may be a viable solution if you want to attract developers to your project. However, every user will need to have a Python distribution installed.

  It seems to me that the barrier to entry, from the point of view of potential plugin writers, is lower if you use scripting than if you choose DLLs.

  Now, back to my initial question: things get much more complicated if you need the plugins to interact with your user interface, e.g. by placing controls on it. In general, DLLs cannot be used to do this. The Borland/CodeGear-sanctioned way is to use packages (BPLs). With BPLs, you can access and instantiate classes offered by a plugin as if they were declared in your host application. The catch is, all BPLs must be compiled with the same exact version and build of Delphi that your main application is. In my opinion this makes packages completely impractical, since it's hard to expect that all the potential plugin writers around the world will be using the same version of Delphi you are. A major pitfall.

  To get around this, I have experimented with a different approach: keep using DLLs, and develop a syntax for the plugin to describe the UI it needs, then create the UI yourself in the host app. (XML is a convenient way to express the UI, since you get the concept of parenting / nesting for free.) The UI description syntax can include callbacks to the DLL triggered when the contents or state of the control changes. This method will restrict plugins to the set of VCL controls your application already uses or has registered, though. And it's not a one-nighter job, while BPLs certainly are.

  Google for "Delphi plugin framework", too. There are some ready-made solutions, but as far as I know they usually use BPLs, with their limited usefulness.

  mghie : +1. Lots of very good advice in here. Creating a plug-in interface has a lot of consequences, for a very long time, so the more thinking about it in advance the better.

• The most universal method of adding plug-in capability is to use COM. A good book to get you started on the road is Delphi Com Programming by Eric Harmon. While it was originally written for Delphi versions 3 through 5, the books contents are still valid with the latest versions of Delphi.

  Personally, I have used this technique along with active scripting to allow end user customization.

• At first I went for BPL and DLL base plugins. And found them hard to mantain.

  If you use BPL system, then you need to match BPL version with EXE version. This includes Delphi updates which can break something. I found out (the hard way) that if I need to include all my plugins with every release, there is no point in having plugins at all.

  Then I switched to plain DLL plugins. But that system just complicated code base. And that's not a good thing.

  While crusing the net I found out Lua embedded script language, and delivered with it. Lua is 150K DLL, embedding bytecode compiler, interpreter and very simple and smart dynamic programing language.

  My plugins are simple lua scripts. Easily mantaind and made. There are premade Delphi examples, so you can export any class or procedure as Lua table or function. GUI or not. For example I had TurboPower Abbrevia in my app for zipping. I exported my zipping class to lua, and all plugins now can call zip('.', 'dir.zip') and unzip(). Then I switched to 7zip and only implemented old class to use 7zip. All plugins work as they did, with support for new zip('.', 'dir.7z').

  I made TLuaAction which calls Execute(), Update(), Hint() procedure from its script.

  Lua allso have it's own plugin system that makes it easy to add funcionality to it. Eg luacom make is easy to use COM automation, luainterface allows calling .net from lua. See luaforge for more. There is Lua IDE made in Delphi, with source.

• I found an article by Tim Sullivan:
  Implementing Plug-Ins for Your Delphi Applications