Support conversion of Delta to HTML and Markdown #15

ArjanAswal · 2021-02-07T17:17:21Z

This Library converts Zeyfr documents to HTML.
There are many plugins that can help with the conversion of HTML to markdown. I was thinking that with some tweaking we can create our own converter too.

Or we can take another approach, we can first convert Delta to markdown like here and then to HTML.
Again, this requires just some minor tweaking.

singerdmx · 2021-02-07T17:26:35Z

It is also possible to do it at server side.
We found a Golang package to do it singerdmx/BulletJournal@16fc1c2

singerdmx · 2021-02-07T17:29:23Z

Note our underlying format is different than Notus

singerdmx · 2021-02-07T17:30:06Z

Zefyr's library won't work since it is missing a lot of attributes.

ArjanAswal · 2021-02-07T17:33:31Z

It is also possible to do it at server side.
We found a Golang package to do it singerdmx/BulletJournal@16fc1c2

This could be a huge limitation for flutter apps looking to use this package.
Any dart script that gets the job done will be very valuable for the flutter developers looking to work with this library.

singerdmx · 2021-02-07T17:35:23Z

You can modify Zefyr's library to submit a pull request if you want.
Note that our data format is 100% sticking to the standard similar to what https://github.com/dchenk/go-render-quill did.

singerdmx · 2021-02-07T17:35:56Z

I mean you can copy Zefyr's code and make changes to it. It is much easier

ArjanAswal · 2021-02-07T17:36:57Z

@singerdmx In my opinion, there are only a few minor changes needed to be made in this code for it to work with this package.

Like replacing NotusAttribute with Attribute, and introducing getters and a few functions for the Style object.

singerdmx · 2021-02-07T17:38:55Z

  Map<String, String> _supportedElements = {
    "li": "block",
    "blockquote": "block",
    "code": "block",
    "h1": "block",
    "h2": "block",
    "h3": "block",
    "div": "block",
    "em": "inline",
    "strong": "inline",
    "a": "inline",
    "p": "block",
    "img": "embed",
    "hr": "embed",
  };

The above is what Zefyr supports.
Yes, for flutter-quill it is https://github.com/singerdmx/flutter-quill/blob/master/lib/models/documents/attribute.dart

ArjanAswal · 2021-02-07T17:40:12Z

You can modify Zefyr's library to submit a pull request if you want.
Note that our data format is 100% sticking to the standard similar to what https://github.com/dchenk/go-render-quill did.

That is why this library is leaps and bounds better than Zeyfr library in my humble opinion.

The above is what Zefyr supports.
Yes, for flutter-quill it is https://github.com/singerdmx/flutter-quill/blob/master/lib/models/documents/attribute.dart

This is right. That is why only a few changes are needed to be made in this code before it can work with this library.

ArjanAswal · 2021-02-07T17:42:39Z

Here is my modified version (Doesn't work yet, needs a few getters and functions from the Flutter-quill library)

import 'dart:convert';

import 'package:flutter_quill/models/documents/attribute.dart';
import 'package:flutter_quill/models/documents/style.dart';
import 'package:quill_delta/quill_delta.dart';

class QuillMarkdownCodec extends Codec<Delta, String> {
  const QuillMarkdownCodec();

  @override
  Converter<String, Delta> get decoder =>
      throw UnimplementedError('Decoding is not implemented yet.');

  @override
  Converter<Delta, String> get encoder => _QuillMarkdownEncoder();
}

class _QuillMarkdownEncoder extends Converter<Delta, String> {
  static const kBold = '**';
  static const kItalic = '_';
  static final kSimpleBlocks = <Attribute, String>{
    Attribute.blockQuote: '> ',
    Attribute.ul: '* ',
    Attribute.ol: '1. ',
  };

  @override
  String convert(Delta input) {
    final iterator = DeltaIterator(input);
    final buffer = StringBuffer();
    final lineBuffer = StringBuffer();
    Attribute<String> currentBlockStyle;
    var currentInlineStyle = Style();
    var currentBlockLines = [];

    void _handleBlock(Attribute blockStyle) {
      if (currentBlockLines.isEmpty) {
        return; // Empty block
      }

      if (blockStyle == null) {
        buffer.write(currentBlockLines.join('\n\n'));
        buffer.writeln();
      } else if (blockStyle == Attribute.codeBlock) {
        _writeAttribute(buffer, blockStyle);
        buffer.write(currentBlockLines.join('\n'));
        _writeAttribute(buffer, blockStyle, close: true);
        buffer.writeln();
      } else {
        for (var line in currentBlockLines) {
          _writeBlockTag(buffer, blockStyle);
          buffer.write(line);
          buffer.writeln();
        }
      }
      buffer.writeln();
    }

    void _handleSpan(String text, Map<String, dynamic> attributes) {
      final style = Style.fromJson(attributes);
      currentInlineStyle =
          _writeInline(lineBuffer, text, style, currentInlineStyle);
    }

    void _handleLine(Map<String, dynamic> attributes) {
      final style = Style.fromJson(attributes);
      final lineBlock = style.get(Attribute.block);
      if (lineBlock == currentBlockStyle) {
        currentBlockLines.add(_writeLine(lineBuffer.toString(), style));
      } else {
        _handleBlock(currentBlockStyle);
        currentBlockLines.clear();
        currentBlockLines.add(_writeLine(lineBuffer.toString(), style));

        currentBlockStyle = lineBlock;
      }
      lineBuffer.clear();
    }

    while (iterator.hasNext) {
      final op = iterator.next();
      final lf = op.data.indexOf('\n');
      if (lf == -1) {
        _handleSpan(op.data, op.attributes);
      } else {
        var span = StringBuffer();
        for (var i = 0; i < op.data.length; i++) {
          if (op.data.codeUnitAt(i) == 0x0A) {
            if (span.isNotEmpty) {
              // Write the span if it's not empty.
              _handleSpan(span.toString(), op.attributes);
            }
            // Close any open inline styles.
            _handleSpan('', null);
            _handleLine(op.attributes);
            span.clear();
          } else {
            span.writeCharCode(op.data.codeUnitAt(i));
          }
        }
        // Remaining span
        if (span.isNotEmpty) {
          _handleSpan(span.toString(), op.attributes);
        }
      }
    }
    _handleBlock(currentBlockStyle); // Close the last block
    return buffer.toString();
  }

  String _writeLine(String text, Style style) {
    var buffer = StringBuffer();
    if (style.contains(Attribute.header)) {
      _writeAttribute(buffer, style.get<int>(Attribute.header));
    }

    // Write the text itself
    buffer.write(text);
    return buffer.toString();
  }

  String _trimRight(StringBuffer buffer) {
    var text = buffer.toString();
    if (!text.endsWith(' ')) return '';
    final result = text.trimRight();
    buffer.clear();
    buffer.write(result);
    return ' ' * (text.length - result.length);
  }

  Style _writeInline(
      StringBuffer buffer, String text, Style style, Style currentStyle) {
    // First close any current styles if needed
    for (var value in currentStyle.values) {
      if (value.scope == AttributeScope.line) continue;
      if (style.containsSame(value)) continue;
      final padding = _trimRight(buffer);
      _writeAttribute(buffer, value, close: true);
      if (padding.isNotEmpty) buffer.write(padding);
    }
    // Now open any new styles.
    for (var value in style.values) {
      if (value.scope == AttributeScope.line) continue;
      if (currentStyle.containsSame(value)) continue;
      final originalText = text;
      text = text.trimLeft();
      final padding = ' ' * (originalText.length - text.length);
      if (padding.isNotEmpty) buffer.write(padding);
      _writeAttribute(buffer, value);
    }
    // Write the text itself
    buffer.write(text);
    return style;
  }

  void _writeAttribute(StringBuffer buffer, Attribute attribute,
      {bool close = false}) {
    if (attribute == Attribute.bold) {
      _writeBoldTag(buffer);
    } else if (attribute == Attribute.italic) {
      _writeItalicTag(buffer);
    } else if (attribute.key == Attribute.link.key) {
      _writeLinkTag(buffer, attribute as Attribute<String>, close: close);
    } else if (attribute.key == Attribute.header.key) {
      _writeHeadingTag(buffer, attribute as Attribute<int>);
    } else if (attribute.key == Attribute.block.key) {
      _writeBlockTag(buffer, attribute as Attribute<String>, close: close);
    } else {
      throw ArgumentError('Cannot handle $attribute');
    }
  }

  void _writeBoldTag(StringBuffer buffer) {
    buffer.write(kBold);
  }

  void _writeItalicTag(StringBuffer buffer) {
    buffer.write(kItalic);
  }

  void _writeLinkTag(StringBuffer buffer, Attribute<String> link,
      {bool close = false}) {
    if (close) {
      buffer.write('](${link.value})');
    } else {
      buffer.write('[');
    }
  }

  void _writeHeadingTag(StringBuffer buffer, Attribute<int> heading) {
    var level = heading.value;
    buffer.write('#' * level + ' ');
  }

  void _writeBlockTag(StringBuffer buffer, Attribute block,
      {bool close = false}) {
    if (block == Attribute.codeBlock) {
      if (close) {
        buffer.write('\n```');
      } else {
        buffer.write('```\n');
      }
    } else {
      if (close) return; // no close tag needed for simple blocks.

      final tag = kSimpleBlocks[block];
      buffer.write(tag);
    }
  }
}

singerdmx · 2021-02-07T17:42:51Z

That is why only a few changes are needed to be made in this code before it can work with this library.

Hope so. We can add this ourself but it is lower priority for us - you may need to wait several months.
Currently we are updating text/background color button in toolbar to show color attributes.
If you cannot wait, you are welcome to submit a pull request.
Let's keep this issue open until we implement it to keep track.

singerdmx · 2021-02-07T22:56:37Z

@ArjanAswal check out https://github.com/memspace/zefyr/pull/381/files

ArjanAswal · 2021-02-08T08:51:49Z

@singerdmx That PR you mentioned works with quill_delta: ^1.0.0 so there were some versioning conflicts but to resolve it I simply added notus to local files.

notusMarkdown.encode(controller.document.toDelta());

The above code doesn't work and shows this error:

The argument type 'Delta (where Delta is defined in C:\flutter\.pub-cache\hosted\pub.dartlang.org\quill_delta-2.0.0\lib\quill_delta.dart)' can't be assigned to the parameter type 'Delta (where Delta is defined in D:\Workspace\Projects\Apps\app\lib\notus\quill_delta.dart)'.

The quill_delta version is different that is the root cause of the problem.

singerdmx · 2021-02-08T09:14:15Z

OK, that is just for reference.
As you know, our underlying data format are quite different and you cannot use notus for flutter-quill.

ArjanAswal · 2021-02-08T10:28:58Z

As you know, our underlying data format are quite different and you cannot use notus for flutter-quill.

This is unfortunately the crux of the matter. I actually found the way to convert Zeyfr document into flutter-quill document and vice-versa. My plan is to convert the flutter-quill doc to Zeyfr doc and then convert it into markdown.

Will keep you posted.

ArjanAswal · 2021-02-08T12:28:30Z

@singerdmx I have successfully created the Delta to markdown converter. Right now it has a few limitations, such as not able to convert 'color', 'background' and 'image' and 'strike' attributes (those attributes that flutter-quill has but Zeyfr doesn't) and leaves them. But it does convert successfully everything else.

It depends on quill_delta : 1.0.0 so how can I file the PR?
Or should I create a package?

singerdmx · 2021-02-08T12:52:18Z

1.0.0 is just one file right?
Just copy it over

ArjanAswal · 2021-02-08T12:56:16Z

1.0.0 is just one file right?
Just copy it over

It also depends on notus and it is quite large.

singerdmx · 2021-02-08T12:58:20Z

Taking dependency on notus is fine.
There is no conflict.

ArjanAswal · 2021-02-08T13:05:56Z

Taking dependency on notus is fine.
There is no conflict.

There is, as notus itself depends on quill_delta 1.0.0.
The best way is to create a package in my opinion.

singerdmx · 2021-02-08T13:07:47Z

Ok it seems so

ArjanAswal · 2021-02-08T18:17:00Z

I have created the quill_markdown package and it converts quill to markdown and vice-versa.
Here is the repo.

singerdmx · 2021-02-08T18:18:03Z

I think it is private repo

ArjanAswal · 2021-02-08T18:26:48Z

However, there are a few limitations.
This package works by converting flutter-quill document to Zeyfr and then converts it into markdown. This is a temporary solution until this library implements some drastic changes to its Style Attributes so that it can be parsed to markdown.
For now, only those attributes that are common to both flutter-quill and Zeyfr are supported. Luckily that covers most of the Attributes.

For markdown to quill conversion, the markdown format needs to be very basic. Any fancy attributes are not supported yet.

Let's keep this issue open till we implement our own conversion code without being at the mercy of Zeyfr.
Fortunately, for now, quill_markdown gets the job done.

singerdmx · 2021-02-08T18:28:17Z

I cannot access this repo

ArjanAswal · 2021-02-08T18:37:25Z

I cannot access this repo

Check now.

singerdmx · 2021-02-08T18:38:15Z

I can access now

singerdmx · 2021-02-08T18:38:38Z

So this is converting to markdown instead of html?

ArjanAswal · 2021-02-08T18:39:17Z

So this is converting to markdown instead of html?

There are existing packages for the conversion of md to HTML.
So you can first convert delta to markdown and then to HTML or vice-versa.
But it all depends on the HTML tags used(whether they are compatible with Quill) and also on the converted markdown attributes as well.

ArjanAswal · 2021-02-08T19:02:38Z

Pub Package link

KrishnaBros · 2021-03-14T16:02:05Z

You guys are working on it seriously or not? Please atleast tell this

ArjanAswal · 2021-03-14T16:04:02Z

No not right now. Feel free to submit a PR if you want. In the meantime you can use the package.

KrishnaBros · 2021-03-14T16:05:55Z

Ok Thanks For Informing

No not right now. Feel free to submit a PR if you want. In the meantime you can use the package.

friebetill · 2021-03-17T22:07:12Z

I created a new converter delta_markdown. It's an adapted version of an existing Markdown - Delta (Zefyr) converter so that it can now convert between Markdown - Delta (flutter_quill). It works in simple cases and I will probably extend it further in the coming weeks. One advantage is that images are supported.

Currently it only works with flutter_quill ^0.3.5, as I am in the process of migrating to Flutter 2.

singerdmx · 2021-03-17T22:37:43Z

@friebetill if it works out, you can update our REAMDE to add a section to use yours

kdela · 2021-04-10T20:38:53Z

In case if someone is looking for workaround

import 'dart:convert';

import 'package:delta_markdown/delta_markdown.dart';
import 'package:flutter_quill/models/quill_delta.dart';
import 'package:markdown/markdown.dart';

String quillDeltaToHtml(Delta delta) {
  final convertedValue = jsonEncode(delta.toJson());
  final markdown = deltaToMarkdown(convertedValue);
  final html = markdownToHtml(markdown);

  return html;
}

friebetill · 2021-05-31T20:48:15Z

As it's possible with the library delta_markdown to convert between delta and markdown and with the library markdown to convert between markdown and html, I close the issue.

If you disagree, please write a comment and I will reopen it.

njovy · 2021-07-25T16:23:52Z

@friebetill You saved my day. Much appreciate it!

Hamza5 · 2021-11-03T18:32:44Z

@friebetill instead of dropping support from many features (like colors) because they are not supported in Markdown, you should allow the user to customize it as he wants. For example, the user may like to encode them as elements with a style, since using HTML inside Markdown is very popular.

mrverdant13 · 2021-11-16T15:35:20Z

For us to be able to create other extensible packages, it would be handy to isolate the delta implementation used for flutter_quill in a separate package.

TarekkMA · 2021-11-22T18:51:15Z

I have created a new library that can convert from and to markdown that addresses the following:

Uses latest version of markdown library.
~95% test coverage with complex full markdown articles and test cases from GFM Spec. Also, I used @friebetill tests from delta_markdown
Support for:
- inline code
- strikethrough
- image
- horizontal rule
Ability to define custom behavior (styles/embeds) (currently I'm working on adding markdown table as an embeddable element in quill)

Contribution and feedback are most certainly welcome.

Library Repo: https://github.com/TarekkMA/markdown_quill
Pub Link: https://pub.dev/packages/markdown_quill

I will release it soon to pub.dev.

Hamza5 · 2021-11-23T08:30:41Z

I have created a new library that can convert from and to markdown that addresses the following:
* Uses latest version of [markdown](https://pub.dev/packages/markdown) library.

* ~95% test coverage with complex full markdown articles and test cases from [GFM Spec](https://github.github.com/gfm/). Also, I used @friebetill tests from [delta_markdown](https://pub.dev/packages/delta_markdown)

* Support for:
  
  * inline code
  * strikethrough
  * image
  * horizontal rule

* Ability to define custom behavior (styles/embeds) (currently I'm working on adding markdown table as an embeddable element in quill)
Contribution and feedback are most certainly welcome.

Library Repo: https://github.com/TarekkMA/quill_markdown

I will release it soon to pub.dev.

Good luck bro,

Unfortunately, I found that Markdown is not the most suitable language for me since it needs a lot of customization, so I switched to HTML because it has all the rich text stuff out-of-the-box. I am currently writing my own code to export to and import from HTML, using the available parser library called html.

TarekkMA · 2021-11-23T08:55:57Z

@Hamza5 thank you, in my current project markdown is a requirement so I will need to stick to it.

friebetill · 2021-11-23T10:24:13Z

Nice @TarekkMA! I will try your version and if it's better than my hacky package, I will link to your package, deprecate my package and archive my repository.

TarekkMA · 2021-11-24T17:07:38Z

Published on pub https://pub.dev/packages/markdown_quill

ArjanAswal added the enhancement New feature or request label Feb 22, 2021

florianh01 mentioned this issue Mar 14, 2021

**Before Closing , Atleast Tell What You guys can do or going to do?** #84

Closed

This was referenced Apr 1, 2021

Question: How to get the plainText with the html tags #126

Closed

How to get html value from Editor ? #130

Closed

singerdmx pinned this issue Apr 10, 2021

friebetill closed this as completed May 31, 2021

friebetill unpinned this issue May 31, 2021

yzxben self-assigned this Oct 6, 2021

ibhanu mentioned this issue Oct 26, 2021

[Mobile] Exception thrown ("targetChild should not be null") when deleting highlighted text #348

Closed

longch mentioned this issue Feb 15, 2022

[Mobile][Android][IOS]Select all, then cut will report error in version 4.0.1 #667

Closed

Repository owner deleted a comment from hmbenhaim Apr 7, 2022

Repository owner locked and limited conversation to collaborators Apr 7, 2022

Support conversion of Delta to HTML and Markdown #15

Support conversion of Delta to HTML and Markdown #15

Comments

ArjanAswal commented Feb 7, 2021 • edited Loading

singerdmx commented Feb 7, 2021

singerdmx commented Feb 7, 2021 • edited Loading

singerdmx commented Feb 7, 2021

ArjanAswal commented Feb 7, 2021

singerdmx commented Feb 7, 2021

singerdmx commented Feb 7, 2021

ArjanAswal commented Feb 7, 2021

singerdmx commented Feb 7, 2021

ArjanAswal commented Feb 7, 2021

ArjanAswal commented Feb 7, 2021

singerdmx commented Feb 7, 2021

singerdmx commented Feb 7, 2021

ArjanAswal commented Feb 8, 2021

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021 • edited Loading

ArjanAswal commented Feb 8, 2021 • edited Loading

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021

singerdmx commented Feb 8, 2021

singerdmx commented Feb 8, 2021

ArjanAswal commented Feb 8, 2021 • edited Loading

ArjanAswal commented Feb 8, 2021 • edited Loading

KrishnaBros commented Mar 14, 2021

ArjanAswal commented Mar 14, 2021

KrishnaBros commented Mar 14, 2021

friebetill commented Mar 17, 2021 • edited Loading

singerdmx commented Mar 17, 2021

kdela commented Apr 10, 2021

friebetill commented May 31, 2021 • edited Loading

njovy commented Jul 25, 2021

Hamza5 commented Nov 3, 2021

mrverdant13 commented Nov 16, 2021

TarekkMA commented Nov 22, 2021 • edited Loading

Hamza5 commented Nov 23, 2021

TarekkMA commented Nov 23, 2021

friebetill commented Nov 23, 2021

TarekkMA commented Nov 24, 2021

ArjanAswal commented Feb 7, 2021 •

edited

Loading

singerdmx commented Feb 7, 2021 •

edited

Loading

ArjanAswal commented Feb 8, 2021 •

edited

Loading

ArjanAswal commented Feb 8, 2021 •

edited

Loading

ArjanAswal commented Feb 8, 2021 •

edited

Loading

ArjanAswal commented Feb 8, 2021 •

edited

Loading

friebetill commented Mar 17, 2021 •

edited

Loading

friebetill commented May 31, 2021 •

edited

Loading

TarekkMA commented Nov 22, 2021 •

edited

Loading