Case Study in Customising Syntax Highlighting and Folding in Vim for a Niche Scripting Language Called Inquisit

To contact us Click HERE

This post presents my efforts to setup a productive environment for editingInquisit scripts in Vim.In addition to being relevant to people who write Inquisit scripts, the postis designed as a general case study in customising Vim for a nicheprogramming language.Specifically, the post discusses how to setup in Vim:(a) code folding using a custom expression,(b) custom syntax highlighting, and(c) interactions with the command-line.

Overview

Inquisit is proprietary software typically used for running computerisedexperiments in the behavioural sciences.The Inquisit Website has further explanationand includes a free limited trial.I have previously discussed the benefits of the software for running onlinepsychological experiments.

Inquisit includes desktop software that runs on Windows.It can be used to write, debug, and run Inquisit scripts that control theflow of the experiment.I have provided a simple example of a script here as a gist on github.It contains a simple RT task, four choice RT task, and a typing test.Further example scripts can be seen in the Task Library on the InquisitWebsite.

More recently I have started using Vim to do my text editing (see myprevious post on this conversion.The conversion was motivated by an aim to increase the power and consistencyof my text editing environment. Thus, instead of using a separate text editor for each programming languagethat I code in, I wanted, as much as possible to be able to use Vimfor as many tasks as possible.

Vim has great built-in support for many languages, however, Inquisit is aniche scripting language.Thus, Vim lacked built-in support for the language.Nonetheless, to true power of Vim is its customisability.

Below I explain the customisations I developed to work more effectively withInquisit.It is still a work in progress, and imperfect. However, even at this stage, I find writing Inquisit scripts far moreenjoyable than using the editor built-in to the Inquisit software.

I present this example as a case study in customising Vim for a newprogramming language.

If you want to follow any updates, check out:

• the git repository with the entire contents of my vimfiles and plugins
• inside the repository is my vimrcwhich includes various customisations for Inquisit file types.
• my syntax highlighting file for Inquisit files

Below I describe a few of these customisations.The sections of greatest general interest are likely to be those on syntaxhighlighting and code folding.

Overview of an Inquisit script

Before describing my customisations, below is a short snippet from anInquisit script:

# general stimuli<text cross>/position=(50%, 50%)/items=("+")/fontstyle = ("Arial", 60pt, false)</text><text error.toosoon>/position=(50%, 50%)/items=("PRESSED TOO SOON")/fontstyle = ("Arial", 30pt, false)/txcolor =(255,0,0)</text>

• The line beginning with a hash is my own internal section marker.All text outside element tags <...>...</...> is not interpreted byInquisit.
• The code is made up of elements which have a keyword name (e.g., text)and an identifier (e.g., cross or error.toosoon).
• In the above example, these elements are used to represent objectsdisplayed on a screen such as a plus symbol or some text.
• Elements have attributes, which have a name and a value.Attributes are contained between opening (e.g., <text ...>) and closingelement tags (e.g., </text>).Attributes begin with a slash, then the name of the attribute (e.g.,position, then equals sign, and then the value of the attribute.
• Values of attributes have various syntactical forms typically involvingeither parentheses or square brackets.

My scripts often range between 500 and 2000 lines of code.For a more complete example, see the gist mentioned earlier.You can even download a free trial of the software if you'd like to see howit runs.The remainder of this post describes the customisations that I developed tomake writing Inquisit scripts in Vim more productive.

Launching an Inquisit Script from Vim

Question

• How can Inquisit Scripts be tested directly from Vim command-line?

Discussion

• Inquisit Help lists command line options

I adopted the following strategy.First, I added the Inquisit executable to the path; E.g., I added C:\Program Files\Millisecond Software\Inquisit 3 to my path.This means that entering inquisit on the command-line starts Inquisit.

Then I added the following code to my vimrc file.

function! RunInquisit()    let cmdstring = "!start inquisit " . '"%:p"'    exe cmdstringendfunction

• Thus, when I have an Inquisit script open in Vim, I just have to type:call RunInquisit() and this will start Inquisit and send the script inthe active buffer to Inquisit for running. Once run, this enables debugging of any errors (Inquisit has a goodbuilt-in debugger). If no errors are obtained, the experiment will start, which facilitatesthe identification of errors in the actual design of the experiment.
• function! declares a new function overriding any previous function bythe same name.
• RunInquisit() is the arbitrary name of the function, although it doesneed to start with a capital letter. In this case, it takes no arguments.
• let cmdstring... assigns a string to the variable cmdstring.The period (.) is the concatenation operator.!start is means of starting a program from the command line asynchronously.inquisit is the name of the Inquisit executable.%:p is a file modifier pattern where % represents the active bufferand :p indicates that the pattern should be replaced with the file namewith complete path for the active buffer. See :h filename-mod for moreinformation.
• exe executes the string on the Vim command-line.

Code Folding

Question

• What is a useful strategy for code folding for Inquisit Scripts?

General discussion of code folding

I'm a huge fan of code folding in general, and in Vim specifically.When done appropriately, code folding provides many benefits.In particular, it makes large files easier to navigate, and it makes iteasier to get an overview of the contents of a file.

After some reflection I have a few principles that I use to guide the designof a folding scheme.

• Use the language
  • Folds should where possible take advantage of natural markers in theprogramming language.
  • Using built-in markers means that time does not need to be allocatedto the task of managing the creation of folds.
• Use unobtrusive markers
  • When markers are used that are not part of the language, they shouldbe simple an unobtrusive.
  • Markers should not accidentally occur.
  • This typically involves taking advantage of the commenting system inthe language.
• Two or three fold levels is typically sufficient
  • Too many folds creates navigational issues whereby excessive time isspent navigating between folds.
  • As a general rule I like it when the lowest level of folding containsat least 5 or more lines and not much more than a page or page and ahalf of text.
• Avoid Fold-1 - text - Fold-2 pattern
  • I.e., when a fold is expanded, there should not be a passage ofexpanded text and then a lower level fold.

Inquisit scripts and code folding

Inquisit scripts are made up of elements that look somewhat like HTML.Good coding practice suggests grouping related elements into sections.This lead me to adopt the following folding scheme.

• Level 1 folds
  • These are lines that begin with a single hash #.
  • As such they are not defined by the language, but the language permitsputting almost any text outside elements.
  • They are designed to represent logical groups of Inquisit elements.
  • I typically have sections for:
    • Experiment: a section that sets out any experiment levelinformation such as global variables, data format, randomallocation of subjects, and so forth.
    • Blocks: a section that sets out each block in the experiment .
    • One for each block: a section that contains all the trials,stimuli, and item code for a given block.
• Level 2 folds
  • These represent each element.
  • Thus, they are automatically created by the language.

My code implementing folding for Inquisit scripts

The code below achieves this design.It is placed in my vimrc file.

function! InquisitLevel(elements)    " elements: 0 or 1    if getline(v:lnum) =~ '^# .*$'        return ">1"    endif    if a:elements && getline(v:lnum) =~ '^<[a-zA-Z].*$'        return ">2"    endif    return "=" endfunction                  au BufEnter *.exp setlocal foldexpr=InquisitLevel(1)au BufEnter *.exp setlocal foldmethod=expr

• It contains a function that takes an argument called elements.If it is 0, then only section headings are folded and not elements.
• getline is a Vim function that returns the content of a line.In this case v:lnum is used by expr fold method to indicate aparticular line number.
• =~ is a logical operator that sees whether the text on the left ismatched by the regular expression on the right.
• '^# .*$' means match from the start of the line (^) where the hashcharacter is in the first column, followed by a space, and then any number(*) of characters (.) followed by the end of the line ( $).
• If such a match does occur, then return ">1" which means that heading 1starts on that line.See h fold-expr for more information.
• a:elements is a variable that contains the value of the argumentelements passed to the function.The a: is a necessary prefix for arguments.In Vim 0 is false and 1 is true.The double ampersand (&&) is the logical AND operator.
• The second match matches lines that begin (^) with the less than signfollowed by at least one alphabetical character and then any number ofadditional arbitrary characters.This provides a simple way that adequately matches for my purposes thestart of Inquisit elements.
• Thus, if elements is non-zero and there is a match, this line is thestart of a level 2 heading (">2").
• If no matches are obtained, a value of = is returned, which tells exprto use the fold level of the previous line.
• The code is set up automatically using the two au commands.
• BufEnter *.exp is an event that occurs when the buffer is enteredthrough file opening or buffer switching.
• *.exp is a file name filter. Thus, the command only works once the filehas a file names.
• foldexpr is a Vim setting that takes a custom function that determinesfold levels.
• foldmethod is a Vim setting that indicates the type of folding to use.

Using code folding

The main keys that I use for code folding are as follows.

• zj an zk
  • to move the cursor down and up between fold headings.
  • This is not needed when text is already folded, but is useful when thecursor is expanded text and I want to get the cursor to the previousor next fold.
  • I often use zjzx to move down and show only the next fold.
• zc
  • to close a fold.
  • I often follow this up with j and k to navigate over folds.
• zx
  • to show only the current fold.
  • It is an awesome command that I use all the time. It functions like magnifying glass zooming in on the code of interestwhile at the same time providing hierarchical context.
• zM
  • to close all folds to level 1.
  • I often use zMzx when folds have been expanded. This then gives theeffect of only showing the active fold.
• zR
  • to expand all folds.
• zO
  • to expand a single fold and its sub-folds.
  • I often use this when a section of text has more folds than isdesirable.

Syntax highlighting

Question

• What is a basic syntax highlighting system for Inquisit scripts?

Basic installation instructions

If all you want to do is make use of my syntax highlighting for Inquisitfiles, follow these steps:

• Copy my syntax highlighting file for Inquisit files into your syntax directory in your custom vimfiles directory.
• Add the following command to your vimrc file au BufEnter *.exp set ft=inquisit
  • It creates a file type called inquisit that then lets Vim know thatit should apply the Inquisit syntax highlighting.
  • Note that it assumes that you are using the standard *.exp fileextension.

Screenshots

The following are some screenshots showing what the folding and syntaxhighlighting look like with a molokai colour scheme.

Image with sections folded

Image of folding with element expanded

Basic image of the syntax highlighting

Overview of writing the syntax highlighting file

Before describing the syntax file in more detail, I should present afew caveats.I am new to syntax files in Vim, and I am far from a master in regularexpressions.I hacked out a syntax highlighting scheme that works for me, adequately.It is far from perfect.But I guess that's a lesson in itself when it comes to using Vim.Vim allows for near infinite customisation, but you have to consider thereturn on investment of any customisation efforts.

In general, when developing the syntax file I had a look through a fewexample syntax files for what I thought might be similar file types.For every command that I did not understand I did Vim help searches.And in general I just fiddled around with a sample Inquisit file open untilI got something that basically worked.For people needing a little more regular expression training, there's agreat Vim-oriented tutorial here.

Discussion of the syntax highlighting file

At the top of the file was the following:

syntax case ignoresyn clear

• This tells Vim that syntax highlighting should be case insensitive
• syn clear clears any existing syntax highlighting applied to the file, although thisis often not required.

The syntax file then contains many syn keyword commands, such as thefollowing:

syn keyword inquisitElementKeyword contained survey syn keyword inquisitElementKeyword contained surveypage syn keyword inquisitElementKeyword contained textbox ...

• Words like survey, surveypage, etc. are all element names and keywordswithin the language.
• keyword is a simple form of matching that matches the keywordexactly without resorting to patterns.
• inquisitElementKeyword is a grouping name which I use later to assigncolours and show where such keywords are contained.
• The contained keyword allows the scope of keywords tobe limited to only situations where they are contained within some othersyntactical structure.

The syntax file then contains code for matching operators, an example ofwhich is shown below:

syn match inquisitOperator contained "||"syn match inquisitOperator contained "&&"

• This involved using the match approach. This was required because the characters in operators are not alphabeticalcharacters.

I then added code to syntax highlight numbers.

syn match inquisitNumber contained "\<[0-9%][0-9%]*"

• The code shows the diverse options that match provides.It matches one or more numbers or the percent sign.The combination of backslash and less-than-sign means that numbers must beat the start of what Vim sees as a word.

Then came some code to represent regions of text

syn region inquisitString start=+"+ end=+"+

• The above code defined a region based on quotation marks.

• The plus sign is used as an alternate delimiter in order to indicate that the double quotes are the actual text used to match the start and end ofthe region.

  syn region inquisitElementBlock start="<..*>" end="" transparent

• The above code represents an Inquisit element.

• The similarity with HTML tags may be apparent.

• The transparent keyword is used to indicate that this merely representsa logical unit and does not impose any actual syntax highlighting.

  syn region inquisitAttributeKeyword contained start="\<" end=">" syn region inquisitAttributeRegion start="\/" end="=" oneline contains=inquisitAttributeKeyword

• The above code creates a quick and dirty way of highlighting attributenames in elements.

• Such attribute names always appear as a word contained between a slash andan equals sign.
• \< and \> match the start and end of a word, but this is contained.
• The second line indicates that the first line is contained in it.
• The oneline keyword ensures that the match occurs entirely on one line.

After some additional code comes the assignment of colours to groups.The following are a few examples:

hi link inquisitElementKeyword Keywordhi link inquisitOperator Operatorhi link inquisitDelimiter Delimiter  hi link inquisitString Character

• Each command creates a highlighting link between the custom groups createdin the script and a common logical highlighting object used by many colourschemes.
• In case you were interested, I use molokai as my default colour scheme.It is a popular dark scheme available from vim.org, and from ThomasRestrepo's blog.

Getting Inquisit Help while in Vim

Question

Inquisit provides clear and well-organised documentation.In Desktop Inquisit, pressing F1 in a script brings up the help for theelement under the cursor.

• How can I make it easy to get help on Inquisit commands while continuing towork within Vim?

Discussion

The simplest strategy is just to keep a copy of the Inquisit help file openwhile coding and type in search terms directly.In the end, I have found this to be quite adequate.However, I record some thoughts and ideas I explored for more automatedapproaches below:

• HH "C:\Program Files\Millisecond Software\Inquisit 3\inquisit.chm" will openInquisit help (at least on my system given my installation path and version).
• Alternatively there is a online version of help: < http://www.millisecond.com/support/docs/v1/index.htm >
• I asked a question on StackOverflowfor more information.
• I installed keyhh.The following command basically did the trick:keyhh -#klink "data element" c:\Program Files\Millisecond Software\Inquisit 3\inquisit.chmwhere "data element" is a hypothetical search term.
• I also found this vimscriptdesigned to assist interaction with chm help files.

Future refinements

In the future I may:

• Improve the syntax highlighting
• Introduce an Inquisit code formatting customisation
• Improve integration with the Inquisit language particularly with regard toattributes
• Develop code that facilitates writing Inquisit scripts, such as sequencesof numbers for trial and block sequences

General lessons Learnt

• Tinkering with the syntax highlighting, folding, and command-lineinteraction in Vim encouraged me to think more deeply about each featureand how the feature could be optimised for my workflow.
• The ability to customise Vim as I have done here validates my decision toadopt Vim. Vim encourages the development of amazing text editing shortcutkeys; the more they can be applied to the languages that I write in, thebetter.
• Investment in customisation pays off both in terms of (a) making the immediatetask of programming in the given language easier, and (b) making it easierto customise other languages in the future.

Additional Resources

• My Conversion to Vim
• Conducting an online experiment with Inquisit