MediaWiki Custom Pygments

You can define your own Pygments lexer for MediaWiki. This allows custom syntax highlighting for languages that are not part of the default set of lexers in the built-in pygments binary.

Installation

Pygments requires python. The system you're running MediaWiki should have access to either python2 or python3.

Replace the built-in pygments binary used by the SyntaxHighlight extension by following the following steps:

1. Download the latest version of Pygments at https://bitbucket.org/birkenfeld/pygments-main. (https://bitbucket.org/birkenfeld/pygments-main/get/2.4.2.tar.gz)
2. Extract it to extensions/pygments
3. Update LocalSettings.php, set $wgPygmentizePath = "$IP/extensions/pygments/pygmentize";.

To add a custom lexer:

1. Create your lexer in the extensions/pygments/pygments/lexers directory
2. Update the mapfile by running Make mapfiles in the extensions/pygments directory
3. Update the list of lexers by running extensions/SyntaxHighlight_GeSHi/maintenance/updateLexerList.php

On CentOS 8, support for python2 ended and there is no longer 'python2' or 'python3' binaries. Instead, it now uses platform-python. Ensure that the pygments script references /usr/libexec/platform-python for pygments to work.

Custom Terminal Lexer

A simple lexer used on this wiki is for the Terminal output. It has the following lexer code which is just a couple regexes.

# -*- coding: utf-8 -*-
"""
    pygments.lexers.terminal
    16:41, 23 November 2018 (UTC)16:41, 23 November 2018 (UTC)16:41, 23 November 2018 (UTC)[[User:Leo|Leo]] ([[User talk:Leo|talk]]) 16:41, 23 November 2018 (UTC)

    Lexers for terminal output

    :copyright:  Leo's stuff
    :license:  Leo's stuff
"""

import re

from pygments.lexer import RegexLexer, bygroups
from pygments.token import Text, Comment, Operator, Keyword, Name, String, \
    Number, Punctuation, Error

__all__ = ['TerminalLexer']

class TerminalLexer(RegexLexer):
    """
    For terminal output

    .. versionadded:: 0.1
    """

    name = 'terminal'
    aliases = ['terminal']

    tokens = {
        'root': [
            (r'^##.*\n', Comment),
            (r'^(\[*[a-zA-Z0-9\@:~\/\-\]*]*[\$#%]{1})', Name, 'command'),
            (r'^(\[+[a-zA-Z0-9\@:~\/\-\]+]*[\$#%]?)', Name, 'command'),
            (r'.*\n', Text)
        ],

        'command': [
                (r'[^\\\n]+', String),
                (r'\\\n', String),
                (r'\n', String, '#pop')
        ]
    }

Troubleshooting

Issue Updating Mapfiles

If you get the following error while updating the mapfiles and you are using Python 2:

# make mapfiles
...
Traceback (most recent call last):
  File "_mapping.py", line 491, in <module>
    module = __import__(module_name, None, None, [''])
  File "../../pygments/lexers/make.py", line 18, in <module>
    from pygments.lexers.shell import BashLexer
  File "../../pygments/lexers/shell.py", line 140
    EXTRA_KEYWORDS = {'srun'}
                            ^
SyntaxError: invalid syntax
make: *** [mapfiles] Error 1

Edit the lexers/shell.py and comment out line containing EXTRA_KEYWORDS = {'srun'}.

Custom Terminal Lexer Issues

Support Anaconda environment prefixes

(base) $ conda deactivate

Support '>' prompts.

> something in python or Rlang console
something

Shell prompt shows partially as command if it contains a space

[root@kube ~]# kubectl get services --namespace=kube-system kubernetes-dashboard
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   ClusterIP   10.108.158.128   <none>        443/TCP         2m36s

Support heredoc

# cat <<EOF > /somewhere/something.txt
[something something]
this should all be grey text.
everything until the line with
a single EOF.
This should be grey too.
EOF
# this should be a command
## and comments should work
# cat /somewhere/something.txt
[something something]
...

Fixed: Shell prompt's leading space should not be part of the command.

# something something
something

Fixed: The ESXi default shell and zsh % shell are not highlighting properly

[root@server:/vmfs/volumes/ee8ff6f0-af4dbc98] sh ./ghettoVCB.sh -f backup_vms

leo@nas:/data/vm% ls
backup_vms  esxi-backup/  ghettoVCB.conf  ghettoVCB.sh*

Fixed: Empty # will cause the next # to be a comment, even if they are on different lines:

# gpart create -s GPT /dev/ada1
# gpart add -b 2048 -s 1953113520 -t freebsd-zfs -l disk01 /dev/ada1
#
# gpart create -s GPT /dev/ada2
# gpart add -b 2048 -s 1953113520 -t freebsd-zfs -l disk02 /dev/ada2
#
# gpart create -s GPT /dev/ada3
# gpart add -b 2048 -s 1953113520 -t freebsd-zfs -l disk03 /dev/ada3