Custom operators

Custom operators are declared like this:

sub infix:<plus>($x,$y) {
    $x + $y
}
say 1 plus 2;

Custom operators

Custom operators are declared like this:

sub infix:<plus>($x,$y) {
    $x + $y
}
say 1 plus 2;

3

Custom operators

Custom operators are declared like this:

sub infix:<plus>($x,$y) {
    $x + $y
}
say 1 plus 2;

3

sub prefix:<@@>($x) { $x * 2 }
sub postfix:<+++>($y is rw) { $y+=3 }
my $z = @@10;
$z+++;
say $z;

Custom operators

Custom operators are declared like this:

sub infix:<plus>($x,$y) {
    $x + $y
}
say 1 plus 2;

3

sub prefix:<@@>($x) { $x * 2 }
sub postfix:<+++>($y is rw) { $y+=3 }
my $z = @@10;
$z+++;
say $z;

23

Custom operators

• All of unicode is fair game

# dot product
sub infix:<∙>(@a,@b) {
  return [+] @a Z* @b
}
say (1,2) ∙ (3,4)

Custom operators

• All of unicode is fair game

# dot product
sub infix:<∙>(@a,@b) {
  return [+] @a Z* @b
}
say (1,2) ∙ (3,4)

11

Custom operators

• All of unicode is fair game

sub circumfix:<⌊ ⌋>($x) {
  $x.floor
}
say ⌊2.4⌋

Custom operators

• All of unicode is fair game

sub circumfix:<⌊ ⌋>($x) {
  $x.floor
}
say ⌊2.4⌋

2

Combining operators

sub infix:<plus>($x,$y) {
    $x + $y
}
sub infix:<times>($x,$y) {
    $x * $y
}
say 1 plus 2 times 3;

Combining operators

sub infix:<plus>($x,$y) {
    $x + $y
}
sub infix:<times>($x,$y) {
    $x * $y
}
say 1 plus 2 times 3;

9

Precedence

is tighter controls precedence

sub infix:<plus>($x,$y) {
    $x + $y
}
sub infix:<times>($x,$y) is tighter(&infix:<plus>) {
    $x * $y
}
say 1 plus 2 times 3;

Precedence

is tighter controls precedence

sub infix:<plus>($x,$y) {
    $x + $y
}
sub infix:<times>($x,$y) is tighter(&infix:<plus>) {
    $x * $y
}
say 1 plus 2 times 3;

7

Also is looser, is equiv

Chaining operators

sub infix:<to-the-power>($x,$y) {
    $x ** $y
}
say 2 to-the-power 3 to-the-power 2

Chaining operators

sub infix:<to-the-power>($x,$y) {
    $x ** $y
}
say 2 to-the-power 3 to-the-power 2

64

Chaining operators

sub infix:<to-the-power>($x,$y) {
    $x ** $y
}
say 2 to-the-power 3 to-the-power 2

64

associativity

We can fix this. is assoc!

sub infix:<to-the-power>($x,$y) is assoc<right> {
    $x ** $y
}
say 2 to-the-power 3 to-the-power 2

associativity

We can fix this. is assoc!

sub infix:<to-the-power>($x,$y) is assoc<right> {
    $x ** $y
}
say 2 to-the-power 3 to-the-power 2

512

associativity

We can fix this. is assoc!

sub infix:<to-the-power>($x,$y) is assoc<right> {
    $x ** $y
}
say 2 to-the-power 3 to-the-power 2

512

Associativity types:
right
left
non
chain (1 < 2 < 3)
list (1,2 X 3,4 X 5,6)

argument types

define subtraction between strings

sub infix:<->($x,$y) {
  $x.subst($y, "", :g);
}
say "house" - "u";

argument types

define subtraction between strings

sub infix:<->($x,$y) {
  $x.subst($y, "", :g);
}
say "house" - "u";

"hose"

argument types

define subtraction between strings

sub infix:<->($x,$y) {
  $x.subst($y, "", :g);
}
say "house" - "u";

"hose"

But

say 32 - 2

argument types

define subtraction between strings

sub infix:<->($x,$y) {
  $x.subst($y, "", :g);
}
say "house" - "u";

"hose"

But

say 32 - 2

3

argument types

define subtraction between strings

sub infix:<->($x,$y) {
  $x.subst($y, "", :g);
}
say "house" - "u";

"hose"

But

say 32 - 2

3

Argh!

multiple dispatch

We can fix this, too.

multi infix:<->(Str $x, Str $y) {
  $x.subst($y, "", :g);
}
say "house" - "u";
say 32 - 3;

multiple dispatch

We can fix this, too.

multi infix:<->(Str $x, Str $y) {
  $x.subst($y, "", :g);
}
say "house" - "u";
say 32 - 3;

hose
29

multiple dispatch

Also works for strings, ints, constants, or any class.

multi sub infix:<->(Str $x, Str $y) {
  $x.subst($y, "", :g);
}
multi sub infix:<->(Str $x, Int $y) {
    $x.substr(0,$x.chars-$y)
}
multi sub infix:<->("escalator","electricity") {
    "stairs"
}
say "catamaran" - "a";
say "catamaran" - 6;
say "escalator" - "electricity";
say 10 - 5;

multiple dispatch

Also works for strings, ints, constants, or any class.

multi sub infix:<->(Str $x, Str $y) {
  $x.subst($y, "", :g);
}
multi sub infix:<->(Str $x, Int $y) {
    $x.substr(0,$x.chars-$y)
}
multi sub infix:<->("escalator","electricity") {
    "stairs"
}
say "catamaran" - "a";
say "catamaran" - 6;
say "escalator" - "electricity";
say 10 - 5;

ctmrn
cat
stairs
5

example

emulate python % operator

multi sub infix:<%>(Str $f, Numeric $n) {
    return sprintf($f,$n)
}
multi sub infix:<%>(Str $f,List $l) {
    return sprintf($f,$l.flat)
}
say 'This is %d.' % 40;
say 'Pi is about %0.2f and e is about %0.2f' % ( π, e );

example

emulate python % operator

multi sub infix:<%>(Str $f, Numeric $n) {
    return sprintf($f,$n)
}
multi sub infix:<%>(Str $f,List $l) {
    return sprintf($f,$l.flat)
}
say 'This is %d.' % 40;
say 'Pi is about %0.2f and e is about %0.2f' % ( π, e );

This is 40.
Pi is about 3.14 and e is about 2.72

example: generating SQL

• DBIx::Class, Rose::DB::Object, SQL::Abtract
• SQLAlchemy, Squeel
• Typical techniques:
  • method chaining
  • operator overloading
  • data structure abuse

example: generating SQL

• DBIx::Class, Rose::DB::Object, SQL::Abtract
• SQLAlchemy, Squeel
• Typical techniques:
  • method chaining
  • operator overloading
  • data structure abuse

Perl 6 operators add new techniques:

SELECT id
FROM user INNER JOIN address ON address.user=user.id
WHERE name = 'ed' and fullname='Ed Jones'

(User + Address)[ name == 'ed' and fullname == 'Ed Jones' ]

• native operators and, == can be defined for columns
• post-circumfix [ ] can be used to filter

example: generating SQL

(User + Address)[ name == 'ed' and fullname == 'Ed Jones' ]

class Table { ... }
multi sub infix:<+>(Table $a, Table $b) {
  ...
}

example: generating SQL

(User + Address)[ name == 'ed' and fullname == 'Ed Jones' ]

class Filter { ... }
class Column { ... }
multi sub infix:<==>(Column $a, $b) {
  ...
  return Filter.new( ... )
}
multi sub infix:<and>(Filter $a, Filter $b) {
  ...
  return Filter.new( ... )
}

example: generating SQL

Relational algebra

• Created by E.F. Codd in 70s
• Mathematical foundation of SQL
• Defines operators such as:
  • Projection (π)
  • Selection (σ)
  • Rename (ρ)
  • Natural join (⋈)
  • Semijoin (⋉)(⋊)
  • Antijoin (▷)
  • Division (÷)
  • Left outer join (⟕)
  • Right outer join (⟖)

example: an ORM

Relational algebra

sub infix:<⋈>($a, $b) {
  "$a NATURAL JOIN $b"
}
say "users" ⋈ "addresses"

users NATURAL JOIN addresses

sub prefix:<∏>(@x) {
  "SELECT " ~ @x.join(', ')
}
say ∏<name age>;

SELECT name, age

Conclusion: Custom operators are useful building blocks for internal informal DSLs.

Slim
http://slim-lang.com

Looks like this:

html
  head
    title Slim Examples
  body
    h1 Markup examples

generates

<html>
  <head>
    <title>Slim Examples</title>
  </head>
  <body>
    <h1>Markup examples</h1>
  </body>
 </html>

Parsing Slim

• A grammar is a collection of regexes (like a class + methods)
• A token is a regex with no backtracking
• A rule is a regex with significant whitespace

grammar slim {
    rule TOP { <line>+ %% <eol>}
    token line   { <indentation> <tag> [ ' ' <text> ]?  }
    token indentation { <indent>* }
    token indent { '  ' }
    token tag    { \w+ }
    token text   { \V+ }
    token eol    { \n+ }
}
say slim.parse(q:to/DONE/);
html
  head
    title Slim Examples
  body
    h1 Markup Examples
DONE

• X %% Y means [ X ][ Y X ]*

Parsing Slim

html
  head
    title Slim Examples
  body
    h1 Markup examples

grammar slim {
    rule TOP { <line>+ %% <eol> }
    token line   {
      <indentation>
      <tag> [ ' ' <text> ]?
    }
    token indentation { <indent>* }
    token indent { '  ' }
    token tag    { \w+ }
    token text   { \V+ }
}

line => ｢html
        tag => ｢html｣
        indentation => ｢｣
line => ｢  head｣
        tag => ｢head｣
        indentation => ｢  ｣
         indent => ｢  ｣
line => ｢    title Slim Examples｣
        tag => ｢title｣
        text => ｢Slim Examples｣
        indentation => ｢    ｣
         indent => ｢  ｣
         indent => ｢  ｣
...

Generating a DOM

• A grammar can be associated with an object, to perform "actions".
• Methods on the object have the same names as the regexes in the grammar.
• When a regex is reached, the method is called.

Generating a DOM

• A grammar can be associated with an object, to perform "actions".
• Methods on the object have the same names as the regexes in the grammar.
• When a regex is reached, the method is called.
• Let's make a class called "DOM" and make an instance.

grammar slim {
  token tag { ... }
  token text { ... }
  token indentation { ... }
  ...
}

class DOM {
  method tag { ... }
  method text { ... }
  method indentation { ... }
}
my $dom = DOM.new;
slim.parse($text, actions => $dom);

Generating a DOM

Algorithm:

1. Make a Node class which has a parent node + child nodes.
   The nodes will be the DOM tree. Also we will need a stack.
2. When you see a tag, push a new node onto the stack.
3. When you see text, set the text of the top node.
4. When you see indentation, pop until the stack size is the level of indentation.
5. Connect parent + child nodes when moving from the stack to the tree.

Tag: push.
Indentation: maybe pop.

The Code

1. Make a Node class which has a parent node + child nodes.
   The nodes will be the DOM tree. Also we will need a stack.

class Node {
    has Str $.tag;
    has Str $.text is rw;
    has Node $.parent is rw;
    has Node @.children;
}
class DOM {
    my Node @stack;
    has Node $.tree;
    method tag { ... }
    method text { ... }
    method indentation { ... }
}
grammar slim {
    ...
    token tag { ... }
    token text { ... }
    token indentation { ... }
}

The Code

1. When you see a tag, push a new node onto the stack.

    method tag($/) {
       @stack.push: Node.new: tag => ~$/;
    }

2. When you see text, set the text of the top node.

    method text($/) {
      @stack[*-1].text = ~$/;
    }

$/ is the match object for the token
~ stringifies

The Code

3. When you see indentation, pop until the stack size is the level of indentation.
4. Connect parent + child nodes when moving from the stack to the tree.

    method indentation($/) {
        while @stack > @$<indent> {
            my $node = @stack.pop;
            with @stack[*-1] -> $top {
              $node.parent = $top;
              $top.children.push: $node;
            }
        }
    }

$/ is the match object for the indentation token
with checks for definedness

The Code

To print, dump a node, and recursively dump children at one more level of indentation.

class Node {
  ...
  method dump(:$level=0) {
    say "<$.tag>".indent($level);
    say .indent($level) with $.text;
    .dump(level => $level+1) for self.children;
    say "</$.tag>".indent($level);
 }
}

.indent on a string prepends spaces
A leading . calls a method on $_

The Code

grammar slim {
    rule TOP { <line>+ %% <eol>}
    token line   { <indentation> <tag> [ ' ' <text> ]?  }
    token indentation { <indent>* }
    token indent { '  ' }
    token tag    { \w+ }
    token text   { \V+ }
    token eol    { \n+ }
}
class Node {
    has Str $.tag;
    has Str $.text is rw;
    has Node $.parent is rw;
    has Node @.children;
    method dump(:$level=0) {
      say "<$.tag>".indent($level);
      say $.text.indent($level) with $.text;
      .dump(level => $level+1) for self.children;
      say "</$.tag>".indent($level);
  }
}

class DOM {
    my Node @stack;
    has Node $.top;
    method tag($/) {
       @stack.push: Node.new: tag => ~$/;
       $!top //= @stack[0];
    }
    method text($/) {
      @stack[*-1].text = ~$/;
    }
    method indentation($/) {
        while @stack > @$<indent> {
            my $node = @stack.pop;
            with @stack[*-1] -> $top {
              $node.parent = $top;
              $top.children.push: $node;
            }
        }
    }
}
my $dom = DOM.new;
my $match = slim.parse(q:to/DONE/,actions => $dom) or die 'no parse';
html
  head
    title Slim Examples
  body
    h1 Markup Examples
DONE
$dom.top.dump;

Output:

<html>
 <head>
  <title>
  Slim Examples
  </title>
 </head>
 <body>
  <h1>
  Markup Examples
  </h1>
 </body>
</html>

Conclusion: Grammars are useful for parsing external informal DSLs.

Perl6::Grammar

To see examples, use --target=parse:

perl6 --target=parse -e "say 'hello, world'"

Perl6::Grammar

To see examples, use --target=parse:

perl6 --target=parse -e "say 'hello, world'"

- statementlist: say 'hello, world'
  - statement: 1 matches
    - EXPR: say 'hello, world'
      - longname: say
        - name: say
          - identifier: say
      - args:  'hello, world'
        - arglist: 'hello, world'
          - EXPR: 'hello, world'
            - value: 'hello, world'
              - quote: 'hello, world'
                - nibble: hello, world

Perl6::Grammar

• Output matches definitions in the rakudo source
• https://github.com/rakudo/rakudo/blob/nom/src/Perl6/Grammar.nqp

grammar Perl6::Grammar {
  rule statementlist  { ... <statement> ... }
  token statement { ... <EXPR> ... }
  token longname { ... <name> .. }
  token name { ... <identifier> ... }
  token identifier { ... }
  ...
}

Making a slang

Let's try to make a slang in which subroutines are declared like:

lambda foo() { ... }

instead of

sub foo() { ... }

Making a slang

Original (Perl6/Grammar.nqp)

token routine_declarator:sym<sub> {
    'sub' <.end_keyword> <routine_def('sub')>
}

• Let's change 'sub' to 'lambda'.

• Deriving from a class and applying a role gives us a way to have a new class with a single method replaced.

• Deriving from a grammar and applying a role gives us a way to have a new grammar with a single token replaced.

Making a slang

but creates a copy of an object, with a role mixed in.

Making a slang

but creates a copy of an object, with a role mixed in.

BEGIN {
  %*LANG<MAIN> := $~MAIN.grammar but role {
     rule routine_declarator:sym<sub> {
       'lambda' <.end_keyword> <routine_def('sub')>
     }
   }
}

Making a slang

but creates a copy of an object, with a role mixed in.

BEGIN {
  %*LANG<MAIN> := $~MAIN.grammar but role {
     rule routine_declarator:sym<sub> {
       'lambda' <.end_keyword> <routine_def('sub')>
     }
   }
}

lambda foo { say 'hello, world' }
foo();

Lexically scoped slangs

If you put your slang in a module:

# lambda.pm
sub EXPORT {
  %*LANG<MAIN> := $~MAIN but role { ...  }
}

Using it is lexically scoped:

{
  use lambda;
  lambda hello { say  'hi' }
}
## lambda bye { say 'bye' }  # illegal!
sub bye { say 'bye' } # ok

Examples

a few slangs on modules.perl6.org

• Slang::SQL -- sql inline in Perl 6
• Slang::Piersing -- allow identifier names ending in ? or !
• Slang::Tuxic -- allow a space betwen sub and parameter list (sub ($x))

future?

• Specified but not yet implemented
• augment slang, supercede slang

augment slang MAIN { ... }
supercede slang Regex { ... }