Improved Scripting Tools (SSL+ and Script Editor)

Discussion in 'Fallout General Modding' started by phobos2077, Feb 7, 2015.

  1. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    I've been working for some time on improving Timeslip's Script Editor, as well as sslc compiler and Anchorite's int2ssl decompiler.
    Many cool new features were already implemented by Timeslip, some didn't work though, so I had to fix them.

    Now I want to introduce some of the new SSL capabilities to all scripters out there.


    SSL+


    There are quite a few features added, so I guess it's a different language now :) (btw. SSL stands for "Star Trek Scripting Language")

    Following features are vanilla-compatible, does not require Sfall to be interpreted by the game:

    • short-circuit evaluation of logical operators: AND, OR
    • python-inspired conditional expressions (ternary operator)
    • break/continue statements
    • for loops
    • increment/decrement operators (++, --)
    • switch statements
    • variable initialization with expressions
    • optional arguments in user-defined procedures
    • procedure stringify operator
    • fixes for using procedures inside arguments to engine functions

    Arrays-related, sfall required:
    • arrays in local variables
    • bracket syntax
    • dot syntax, chaining
    • array expressions: lists and maps
    • foreach loops

    You can read all about array syntax here. Only recent addition to that stuff was a chained array dereferencing using a sequence of dots and brackets. This way you can use multi-dimensional arrays easily.

    Now some details about most notable (IMO) features.


    Short-Circuit Evaluation

    Answer to "what the hell is this?" can be found here. In Fallout scripts, unlike most languages, logical operators AND, OR were compiled just like any other arithmetic operator. That means every part of such expressions was evaluated, even when result is already known.
    Why this is important? It's simple, consider this real example from Restoration Project code, dude_obj script:
    Code:
    if (cur_map_index == MAP_NEW_RENO_1 and (map_var(MVAR_Kitty_Drug) > 0) and (map_var(MVAR_Kitty_Drug) < 4)) then ...
    
    What is wrong with this code? Nothing, it looks pretty logical. However, it breaks the script and produces hundreds of errors every minute in debug.log. Why? Because all parts are evaluated, even when current map is not New Reno, it still calls "map_var", which fails with message "map var out of range". The problem is in the compiler itself.
    This new feature fixes this and makes such conditions work like you expect. It is currently optional in compile.exe and disabled by default. It can be enabled by -s parameter (or a checkbox in latest Script Editor settings dialog).

    As a side effect of how this works, AND, OR operators now return one of the arguments as a result of expression, instead of always 1 or 0:
    Code:
    y := 0;
    x := y or z; // x will hold the value of z, because y is 0
    

    Conditional Expressions

    Because it was taken from python, you can read everything here. It works exactly the same in SSL+, providing you with a little convenience and another option to branch code execution within expressions.


    Break/Continue statements


    Another important thing, IMO. I really missed this on several occasions, even added "foreach (x in y while z)" syntax in foreach loops, just because there were no break. Later I discovered that it is possible, and pretty easy to implement, using existing fallout scripting opcodes.

    Example:
    Code:
    for (obj := tile_get_next_critter(tile, elevation, 0); obj != 0; obj := tile_get_next_critter(tile, elevation, obj)) begin
        // do some exciting stuff
        if (did_all_I_wanted) then
            break; // exit from loop prematurely
    end
    
    NOTE: continue, like in other languages, will properly respect increment operator in for and foreach loops so it will be executed before next iteration starts.


    "For" Loops

    Well, one example of such loop is above. Should be pretty obvious for everyone who used C++/C#/PHP/JavaScript or similar language. It was originally made by Timeslip, I only fixed a couple of bugs related to them and made it possible to use parentheses around parameter list.


    "Switch" Statements

    Nothing new on my part here. I never honestly used them in my scripts, but still they might improve your code when you need to compare one value against large number of constants:
    Code:
    switch (obj_pid(obj_being_used_with)) begin
        case PID_KNIFE: display_msg("You hit it with your knife to no result");
        case PID_CROWBAR: display_msg("You hit it with your crowbar. Still, it won't budge.");
        case PID_SLEDGEHAMMER: display_msg("You hit it with your hammer. Not a scratch. What's wrong with this thing?");
        default:
           display_msg("This won't help here.");
    end
    

    Variable Initialization with Expressions

    I find it very useful when writing hook scripts for sfall:
    Code:
    procedure keypress_hook begin
       variable pressed := get_sfall_arg,
                key := get_sfall_arg;
     // ...
    end
    
    Previously it would fail to compile. You can use any complex expressions here. Only restriction - if it starts with a constant number or a string, you will need to enclose it in parentheses.


    Optional Arguments

    Another thing common in other languages that I missed. It works only for user-defined procedures. You need to specify default values for arguments so they could be omitted. Default values should be simple constants. Doesn't work for inline functions (not that anyone cares to use them anyway :D )
    Code:
    procedure myproc(variable a, variable b := "sdsd", variable c := 0) begin
       display_msg( a + b + c );
    end
    
    // someplace else
    call myproc(5); // valid
    call myproc(7, "no"); // valid
    call myproc(); // not valid, need at least 1 argument
    
    How this works: it basically puts the constants you provide in place of omitted arguments. No fancy tricks here.


    Fixes for procedures inside arguments to engine functions

    This is relevant if you have a user-defined procedure without arguments and let's say you want to call it like this:
    Code:
    NOption(get_line_number, Node002a, 004);
    
    What you wanted is to call this function so it return line number, but instead, compiler will place this function ordinal number into the compiled script. Good luck finding this bug.
    With new compiler, only procedures at appropriate argument positions will be treated as procedure references (second argument in above example). Also, decompiler will place empty parentheses after procedures call, so it will be more obvious what has compiled:
    Code:
    giq_option(4, 205, get_line_number(), Node002a, 0);
    


    Procedure Stringify Operator

    Basically it puts string with procedure name in it's place. So @Node006 will be compiled as "Node006". As simple as that.
    Now why the hell do we need it? Well, in fallout scripts you can call procedures by their names contained in a string. You can pass this string around and call it from other places. This basically a ghetto version of passing functions by reference (sadly, no anonymous functions... yet).
    This operator helps to use this feature better, as it:
    • prevents optimizer from deleting your procedure, because it will be properly referenced
    • allows to see such procedure references in Script Editor
    Code:
    procedure get_name(variable a) begin
       if not(a) then return "NULL";
       return obj_name(a);
    end
    
    procedure array_map_func(variable arr, variable callback) begin
       variable k, v, i;
       foreach (k: v in arr)
          arr[k] := callback(v);
       return arr;
    end
    
    // this will display names of all party members
    display_array( array_map_func(party_member_list) );
    //          (display_array is a user procedure which print array contents)
    

    Again, all this features were done using existing low-level scripting operators. Scripts compiled with them will work on any Fallout version, even on new engine (if it will properly implement all basic opcodes).
    int2ssl was updated to support all this syntax (along with sfall opcodes), to properly decompile new scripts.



    Script Editor


    There were numerous small-to-medium fixes and improvements, so I won't list all of them, only the big and recent ones.

    Parser

    It shows a list of all procedures and global variables in current script. This list can be found in the right part of the main window. Clicking on an item will bring you to it's location in code.
    This is Timeslip's work, but I've updated code parser to support all latest syntax and sfall scripting functions and tried to make it look somewhat more friendly.


    Hover Tooltips

    Hovering over a scripting function, will bring you help tooltip like this:



    Code Completion

    As you write function name, a list of suggestions will show up, you can browse it with arrow keys and select with Enter:



    It still ugly and primitive, but I find it extremely helpful. Not only to see names of scripting functions (not having to open reference every time), but also to see constants like PID_LEATHER_ARMOR. Don't guess the name, don't open define.h, just start typing and select it.


    Hope this will inspire someone to start/continue scripting.


    Script Editor with latest compiler/decompiler: download v3.5.4.3
     
    Last edited: Feb 7, 2015
    • [Like] [Like] x 6
  2. JimTheDinosaur

    JimTheDinosaur Vault Dweller
    Modder

    Mar 17, 2013
    Absolutely amazing work phobos. Especially the short-circuit evaluation is something that should probably be implemented in every mod currently in existence. Unless everybody except for me was aware of AND not working in this way already and didn't take the necessary precautions of course :P.
     
  3. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    Well the fun part is that you don't need to do anything to "implement" it into the mod, only to recompile with -s option. The only thing that should be considered is return value of such expressions. Example:
    Code:
    Y := 111;
    Z := "apple";
    X := Y and Z; 
    
    // with original compiler, X will be 1, so
    if (X == true) // will be positive
    
    // but with new behavior, X will be "apple". So comparing it to "true" as above won't work. Though I doubt someone will do such a thing anyway.
    
    PS: I just tested the same in JavaScript and Python 3. Works exactly the same! I guess this makes it a "correct" behavior :D

    One last thing to consider - priority of ANDs and ORs. In all languages, AND has higher precedence, but in SSL they are equal. Though I had a hard time coming up with an example of where this might be a problem, I can change this relatively easily. Though I personally don't care at this point :)
     
    Last edited: Feb 7, 2015
  4. darkf

    darkf Caller of the Void Modder

    Jul 12, 2014
    Nice! Good work. Are you going to release the source / is it available?
     
  5. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    Of course. Script Editor is a part of sfall project. You can find latest sources of sslc/int2ssl in my signature.
     
  6. trollworkout

    trollworkout It Wandered In From the Wastes

    Aug 1, 2015
    phobos amazing work.

    i have a question regarding sfall. is it possible to change the text for the appearance mod let's say from RACE to KIND?
     
  7. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    This is not the appropriate thread to ask, but use translations.ini:
    Code:
    
    [AppearanceMod]
    RaceText=Race
    StyleText=Style
    MainButtonText=Appearance 
    
    
     
  8. trollworkout

    trollworkout It Wandered In From the Wastes

    Aug 1, 2015
    awesome! thank you.
     
  9. burn

    burn Look, Ma! Two Heads!
    Modder

    Apr 22, 2012
    I'm getting some strange behaviour, which I can't explain.
    1) Take Myron's script from RP 2.3.3 (probably works with vanilla as well, but I tested on RP)
    2) Decompile it
    3) Compile it back - without changing anything
    4) Put into data/scripts

    Now, when Myron is asked to re-join the party, he will say that it's full (even if it's not) and decline. But he'll join, anyway. So, actual functionality is not affected, but the dialog gets bugged.
    I narrowed it down to Node1100, but no further.
     

    Attached Files:

  10. Lexx

    Lexx Lived Through the Heat Death
    Moderator

    Apr 24, 2005
    I am a shitprogrammer, otherwise I would do it myself... but as I am incapable of any real programming, I'll dig up this thread to post my suggestion for a Sfall Script Editor expansion.


    For a long time I've got a dream, and the dream is called: Flow Chart Scripting. Especially when it comes to Fallout, it would actually be pretty possible to present the scripts in a simple, easy to overlook flow chart. As the Sfall Script Editor already has everything a Fallout scripter usually needs, such a feature would be a great bonus.

    How would this work?
    You run the editor, open a script, press a button on the interface and BAM, you switch to the flow chart view. In this view, every procedure is shown as a unique box in which you can read and edit the script.

    Why?
    If you'll have to ask "why", then you aren't a dialogue writer. Right now, and ever since, it is really unfun to write dialogue for Fallout 1 / 2. You basically have to work in two files at once- the actual text file, and the script file. Also it's really annoying to write branching dialogue in a linear file format. We are living in the future- nowadays we can do such stuff in a fancy looking flowchart editor. By god, think about the dialogue writer!

    It's also working as a kind of finite state machine: The game is passing through the script in a linear form, which can be more easily comprehended in a flow chart.

    But what about the super old FMF Dialogue Editor? It already exists!
    Yes, but it's clunky, old, and afaik it never really worked that well. Besides, it aint a flow chart dialogue editor! And it works completely different. Its target was to make it possible to "click together" a *script* (with dialogue) without having to know any scripting. My idea is different: It still requires you to script, it just presents the script in a visual pleasing format. It helps scripters to structure the script, and it helps writers to actually see the NPC dialogue. Oh, and it helps with debugging- wrong nodes, missing nodes, etc. etc.

    How would this look like?
    See my mockups:
    See all the script procedures on a page. Easily move them around (drag&drop), write the real script in them. (Node001 has a bug in this example... just ignore it :>) The box recognizes if you write something like "Reply(x);" or "call x", or "NOption..." and changes the format, connects to a different node, etc. basically a more "advanced" syntax highlighting--- makes it easier to read. To connect a procedure block with another one, there actually exist only two possible ways: Either with "call x", or via a dialogue option. Both can be checked by syntax and then converted to a "connector".

    To make it super fancy, it should be possible to display the text line behind the script commands. This way it will be easier to "see" where we currently are in the dialogue:
    ^ Not 100% necessary, but would be fancy.




    So, yeah... Just a few thoughts. Would be awesome to get something like this some day. Preferably before the release date of Fallout 10, when nobody ever looks back at Fo1/2 again. :/

    /Edit: Actually I would pay a programmer to do this. Anyone feeling up for the task?
     
    Last edited: Oct 7, 2016
    • [Like] [Like] x 3
  11. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    Really good idea Lexx. But it will require some real effort to do :)
    The one-way sync from the flow-chart to code is easy, everyone can do that. But two-way sync is much harder, you need to use the script parser and analyze it's output. Right now it's clunky. Basically parser DLL for script editor is compiled from the same code as SSLC and exposes some functions to the outside. So we will need to define statements that link "nodes" together, parse the code for each procedure and analyze these links.

    The most time consuming part would probably be the UI. Need to spend some time to find appropriate control for WinForms, doing this flowchart control will be very difficult probably.
     
    • [Like] [Like] x 1
  12. Lexx

    Lexx Lived Through the Heat Death
    Moderator

    Apr 24, 2005
    Maybe two-way sync isn't necessary, if the flowchart editor has 100% the same capabilities as the regular linear text editor (so you don't feel the need to get back from the flowchart). Also I thought making it an addition to the Sfall editor and not a complete new tool would make it easier / faster to develop, as the Sfall editor already has everything (or most of) what is needed for scripting (I am just missing a global variables editor, duh).

    About the ui- aren't there available libraries for such stuff? Though, like I've said, I am a noob programmer and have no idea.
     
  13. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    I don't know, need to investigate. This editor was created first by Timeslip back in 2010, but now used technologies are outdated (Win Forms) so I'm not sure if there is free and decent libraries available for such UI.

    Anyway, the editor is open source and anyone is free to extend it :)

    Edit: it seems WinForms is still supported by MS. Only need to find decent flow-chart control or build one manually.
     
    Last edited: Oct 13, 2016
    • [Like] [Like] x 1
  14. Lexx

    Lexx Lived Through the Heat Death
    Moderator

    Apr 24, 2005
    For me personally, the most important factor would be to make it easier to write and view the dialogue bits in the script. Hell, if we could write the actual text right inside the script and it's getting written into the msg file automatically, that would be even better. Though I'll doubt it's easily doable.
     
  15. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    Did a bunch of refactoring today. Hopefully now the code is a little more readable and maintainable (main form's class is still 1500 lines though...).
    Also fixed some file-related bugs:
    • Opening files via explorer (command line) in various ways (absolute paths, relative paths, different working folder) should work correctly now.
    • Forced application to single instance. If you open another instance passing file names in command line, those files will be opened in the current application instance instead.
    Github repo
     
  16. burn

    burn Look, Ma! Two Heads!
    Modder

    Apr 22, 2012
    Any insight on this?
     
  17. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    Which version of compiler/decompiler are you using?
     
  18. burn

    burn Look, Ma! Two Heads!
    Modder

    Apr 22, 2012
    The one that was current at the moment. I believe it's 2015.02.08
     
  19. phobos2077

    phobos2077 Mildly Dipped
    Modder

    Apr 24, 2010
    Try compiler from latest sfall modderspack, and this int2ssl. Compiled just now after fixing some bug which produced uncompilable SSL (related to line breaks in string literals).
    Also try compiling without any optimization.

    Binary code of Node1100 seems identical in both versions of script (your attachment and the one in RP).
     
  20. Lexx

    Lexx Lived Through the Heat Death
    Moderator

    Apr 24, 2005
    Anyone ever felt motivated to take a look at that flowchart stuff? I'm about to bite a piece out of my desk, because I hate writing dialog in Notepad just so much by now. In fact, I hate it so much that I even considered looking for tutorials on how to make my own thing... but even the thought gives me the creeps, because all the time I'd spend on this instead (with an uncertain outcome... none, most likely), I'll lose working on my mod. :/