Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eval is run on blog posts? #541

Closed
dnmfarrell opened this issue Sep 12, 2017 · 5 comments
Closed

eval is run on blog posts? #541

dnmfarrell opened this issue Sep 12, 2017 · 5 comments

Comments

@dnmfarrell
Copy link
Contributor

% statocles -v
Statocles version 0.085 (Perl v5.26.0)

blog/2017/09/09/first-post/index.html:

---
title: Plotting With Perl 6
author: Moritz Lenz
tags:
  - inline-python
  - matplotlib
  - git

---

```
my $proc = run :out, <git log --date=short --pretty=format:%ad!%an>;
my (%total, %by-author, %dates);
for $proc.out.lines -> $line {
    my ( $date, $author ) = $line.split: '!', 2;
    %total{$author}++;
    %by-author{$author}{$date}++;
    %dates{$date}++;
}
```

Running statocles build:

Error in template: Error in template: Global symbol "$author" requires explicit package name (did you forget to declare "my $author"?) at /blog/2017/09/09/first-post/index.html line 7.
syntax error at /blog/2017/09/09/first-post/index.html line 7, at EOF
  (Might be a runaway multi-line "" string starting on line 1)
Global symbol "$author" requires explicit package name (did you forget to declare "my $author"?) at /blog/2017/09/09/first-post/index.html line 8.
Global symbol "$date" requires explicit package name (did you forget to declare "my $date"?) at /blog/2017/09/09/first-post/index.html line 8.
Global symbol "$date" requires explicit package name (did you forget to declare "my $date"?) at /blog/2017/09/09/first-post/index.html line 9.
syntax error at /blog/2017/09/09/first-post/index.html line 9, at EOF
2: ```
3: my $proc = run :out, <git log --date=short --pretty=format:%ad!%an>;
4: my (%total, %by-author, %dates);
5: for $proc.out.lines -> $line {
6:     my ( $date, $author ) = $line.split: '!', 2;
7:     %total{$author}++;
8:     %by-author{$author}{$date}++;
9:     %dates{$date}++;
10: }
11: ```
176: 
177:         $content = $t->process( \%args );
178:     }
179: 
180:     if ( blessed $content && $content->isa( 'Mojo::Exception' ) ) {
181:         die "Error in template: " . $content;
182:     }
183:     return $content;
184: }
185: 
186: # Build the Perl string that will unpack the passed-in args
@preaction
Copy link
Owner

The content is run through the Mojo::Template template parser to allow plugins like highlight which does code syntax highlighting. The parser treats % as the first non-whitespace on a line as a single-line template directive. So line 7 (%total{$author}++;) gets parsed as Perl 5 code.

The most direct way to fix this is to double up the % to get Mojo::Template to ignore it (so %%total{$author}++;). Here's the very basic template syntax.

Using the highlight plugin would also avoid this issue, since it'd be treated as highlighted code and not template to parse:

%= highlight perl6 => begin
use v6.c;
%hank<name> = "Hazel Murphy";
say %hank<name>;
% end

That should work, but I've never tested the Perl 6 syntax highlighting before so I don't know how good it looks. This is handled by Syntax::Highlight::Engine::Kate, which is the only good backend highlighter for Perl I could find. I could extend the highlight plugin to use highlight.js or something if needed.

Unfortunately, since you're doing Perl 6, I think you're going to come across this a lot. I could also come up with a way to either disable document template parsing or change what the directive leader looks like. Changing the template directive marks would be more difficult, since a lot of the good ones are taken by Markdown (#, !, = off the top of my head) or could still be confused for source code.

@dnmfarrell
Copy link
Contributor Author

Thanks for explaining the issue. Selfishly, I'd like to be able to type:

``` prettyprint
$foo
```

Or:

``` perl
$foo
```

Or nothing for straight up pre:

```
% cp /foo/bar .
```

And Statocles would produce the appropriate HTML for prettify.js or highlight.js. It does strike me as a negative that Statocles can't parse markdown in some cases because of the template/plugin system.

On the other hand, I'm sure there are benefits to using embedded Perl templates and plugins, so I can't say for sure if this "should be fixed" or not. It feels like a design trade off, that you could make a case for either way.

I don't know how much work it would be, but could the template/plugin system be made optional?

@preaction
Copy link
Owner

I can allow the content template parsing to be disabled, yes. Likely I will make it a setting that could be added at the site level for all documents, at the application level for a single blog's documents, or at the document level (in the frontmatter) for a single document.

The biggest benefit the content templates give Statocles is including files. Most of the additional template functions Statocles currently has are built around that:

%# Include a perl script and highlight it
%= highlight perl => include 'example.pl';

%# Include a markdown file and render it to HTML
%= markdown include 'recipe.markdown';

Template includes are processed as templates and have their own variables, so one could make an "author bio" template and pass in the author's information from the document. (Funny enough, there's a way to disable template processing of an included file: %= highlight perl => include -raw, 'example.pl'; so adding a way to disable it for other content makes sense).

There are the things that Markdown makes tedious. Jekyll has ways of turning data structures into content, and this is something I also needed. This gallery page is generated from a data structure.

The last thing that the template stuff is used for is things that Markdown makes impossible/terrible. Images and other media is a big one. Of course, one is simply supposed to use HTML for these things, but there are other tasks one must do when preparing images for the web, including resizing, resampling, creating thumbnails, preparing different sizes for different browsers, and etc... (this plugin is described in #498).

Right now, I'm pretty sure Text::Markdown does not work with the fenced code blocks (```), and that's the Markdown parser that Statocles uses by default. I'm starting to have the opinion that I should move to CommonMark (which does include fenced code blocks, and even an "info string" which would then let me add the highlighting automatically like you mentioned), but @djerius and I have also been talking of ways to make non-Markdown documents, and making document handling more flexible might open up the tools for dealing with things like templates. If I were to move to CommonMark, the only available Perl module for CommonMark is an XS module, so I'd likely end up having to write the pure-Perl version.

But, so you can keep going, I can make a way to disable the template processing step which happens before the document is given to the Markdown parser.

preaction added a commit that referenced this issue Sep 14, 2017
This attribute can be set on documents, applications, and sites to
enable/disable content template processing.

This isn't an ideal solution, but it is an expedient one. In the future,
we might make the content templates more configurable (allowing users to
set which characters introduce template directives, or which template
processor to use, for example).

Refs #541
@preaction
Copy link
Owner

Okay, v0.086 is uploaded to PAUSE, and allows you to add disable_content_template: true to your site.yml configuration file (in the site object) to disable content template parsing for all documents, or you can add it to the frontmatter to disable it for only a single document (or you can add disable_content_template: false to re-enable it for a single document).

I'm not sure I'm fully pleased with the name or the solution, but if I come up with anything better, this will remain for backwards-compatibility (the burden of all software projects...).

@preaction
Copy link
Owner

I've now also added the ability to configure the template directives, so you can completely disable the line-based code snippets, and change the tags to look like Template Toolkit tags:

site:
    args:
        theme:
            path: '::default'
            tag_start: '[%'
            tag_end: '%]'
            line_start: "\x1F"

This is probably all I'm going to be able to do to help with this.

preaction added a commit that referenced this issue Jun 21, 2018
When we're writing blog posts about Perl, we run afoul of the template
parser, a lot. The `%` that indicates a line of template is the same as
the hash sigil, which often appears on the start of a line.

Now, users can disable line-based processing entirely if they want, and
change the template tags to look however they want (maybe to match
Template Toolkit, so `[% ... %]`).

Refs #541
preaction added a commit that referenced this issue Jun 21, 2018
When we're writing blog posts about Perl, we run afoul of the template
parser, a lot. The `%` that indicates a line of template is the same as
the hash sigil, which often appears on the start of a line.

Now, users can disable line-based processing entirely if they want, and
change the template tags to look however they want (maybe to match
Template Toolkit, so `[% ... %]`).

Refs #541
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants