[Suggestion 4RPL] A custom embeded preprocessor

[kajacx]

This was originaly proposed by KubeRoot on Discord, so I'll try to re-iterate what se said and offer my own observations. The issue is that no matter how many features does V add to 4RPL, it will never be perfect for everyone, since someone will always have something that they would like to add/change. This could be any range of things, like local variables, hexadecimal constants, multi-line comments, script including or better notation for creating and accessing lists.

The explaination

Before I explain any further, let's take a deeper look at how exactly compilation works in PRPL. You start with the PRPL source, and end with a list of compiled commands. But there is already a "preprocessor" in PRPL, it for example handles warp notation, list indexers and attribute setters/getters. Examples:

Code Select Expand


10 pow (<-distance -2 mul)            ->         10 <-distance -2 mul pow
<-list[I]                             ->         <-list I GetListElement
->list[I]                             ->         <-list I SetListElementRPN
<-sid.ShipEnergy                      ->         <-sid GetShipEnergy

(BTW you can look what exactly does the PRPL preprocessor do with my debugger, because the current line is expanded into fully compiled commands.)

So, the aim now is to make it possible for the mapmakers to add "rules" like these themselfs. How? By writing the preprocessor and saving it with the map. You could write the preprocessor in JavaScript for example, or in any other "normal" programming language, as long an CW4 could run that with every compilation. For example, if I wanted local variables, I could easily write a JS code that takes the source code and replaces +>varname with ->unique_prefix__varname to make function-local variables.

Technical challanges

So far I can see 3 major technical challanges in making this work:

• Running a precompiler at all - Basicly you need to take the source code of the precompiler, run it as a program, and feed the original 4RPL source code into it, and then feed the output into the normal 4RPL compiler. Sound a bit complicated, but the only real challange will be executing the preprocessor. Noje.js is probably the way to go for JavaScript.
• Security risks - You don't want the program to do anything crazy with your files for example - neither on purpouse or by accident. A proper way of securing the preprocessor so that it cannot damage your computer is an important part that shouldn't be skipped.
• All the stuff around - Do not underestimate the amount of work that still needs to be done after it technicly "wroks" as a proof of concept. You still need to save the preprocessor code with the map, give the user an easy way to edit it, allow multiple preprocessors to co-exists with settable order, etc.

Examples of usage

Here I list some examples what this could be used for, feel free to post more in the comments.

• Local variables - easly the first thing I would go to. Use +>varname to write to and <+varname to read from a function-local variable. Compiled as ->prefix__varname and <-prefix__varname, with "prefix" being unique for each function.
• Multi-line comments - For example use the C-style /* ... */ comments and then compile it by adding # to the entire block.
• Better list creation - For example write { 1 2 3 } to easily create a list and compile as "_list_start_" 1 2 3 @_create_list_ where _create_list_ will be added as a function to the end of the script that will put items from a stack into a new list until it finds a "_list_start_" string.

These are just a few of the limitless posibilities that this could have. Please note that this could only add "syntax-sugar" style improvemnts, at the end of the day, you still have to generate a regular 4RPL code, so for example this approach couldn't be used to add ShipSpawner functions in PRPL.

Other issues

As great as this sounds, there are unfortunately quite a few problems with this. First, just adding some of the more popular requests into 4RPL outright (like warp notation or list idnexers are in PRPL) would be probably faster than to implement a custom preprocessor like this.

Also, the code could be a bit confusing, with every mapmaker potentionaly using a different version of 4RPL. Of course both the original and preprocessed 4RPL files would be viewable, but the preprocessed one would by expectation look a bit "ugly", making script sharing a bit more difficult. However the mapmakers usually know what they are doing, so I don't think this will be that big of an issue.

We can just use a 3rd party preprocessor. This is the solution that I currently use for PRPL, with PRPL+, that has (you guessed it) local variables. But let me tell you, 3rd party preprocessors aren't optimal. For once I have to start an aditional program when I want to write code, and that's not fun. Also sharing this is a bit of an hassle, requiring other people to download another program just so they can use more features in PRPL is not so that great eighter.

Conclusion

I don't think this will be implemented, and that's probably fine. As I said, jsut taking the more popular requests and adding them into 4RPL directly would mostly eliminate the need for something like this and would be way simpler as well.

Also I've heard rummors that the code for 4RPL will be re-made from the ground up (not like in PF, where is was so blatantly copy-pasted that the internal files were still named CrplCore and CrplCompiler), so that wouldbe a great opportunity to add new features in, however I wouldn't  blame V if he ended up copy-pasting the code again to save time.

[Karsten75]

Great write-up of an intense debate, will need some study on this..

[kubinator4321]

Good writeup, but I guess I'll add a couple things:

1. Node.js might not be the best idea. I don't know enough to judge well, but it has a lot of unnecessary features (file access, sockets, etc) that would bloat the filesize and might require extra work to disable
2. I have made an example implementation as a mod for Creeper World 3, opensource on gitlab

The mod isn't particularly good, it's more of a proof of concept, but it utilizes an opensource library written in C# for running JS and shouldn't have any issues with multiple preprocessors (order defined as appears in .crpl files)

Last, but not least, this is what I'd consider one of the most compelling points of a user-scripted preprocessor: no matter how much time has passed since the game's release and whether the dev is even still a game dev, users would forever be able to tweak and enhance the scripting language syntax however they like. Right now the best you can do is either write an external preprocessor, or ask the dev for features.

[Karsten75]

Last, but not least, this is what I'd consider one of the most compelling points of a user-scripted preprocessor: no matter how much time has passed since the game's release and whether the dev is even still a game dev, users would forever be able to tweak and enhance the scripting language syntax however they like. Right now the best you can do is either write an external preprocessor, or ask the dev for features.

Tweaking and enhancing the scripting language is hardly the point of the game. Right now, no matter how much time has passed, the game still allow mapmakers to make fun, interesting maps. That is more to the point than allowing modifications to the scripting language. The only time the game will become functionally obsolete is when the platform infrastructure becomes unsupported. When that may be is anyone's guess, but there is nothing on the horizon that indicates this will happen.

Right now, all the scripts ever written for the game is available in the same, uniform language. Anyone wishing to modify them has the opportunity to study those scripts, modify them and make alterations. They would not, additionally have to figure out the syntax of the scripting language, worry about whether scripts from various maps are compatible, or whether the script translator would have to be modified.

Finally. The potential audience for alternative script translators is a tiny fraction of the entire game's universe. In fact, it's probably a tiny fraction (-one-) of all the potential mapmakers.