Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dumps String with newlines to Multiline? #236

Open
simplesteph opened this issue Mar 9, 2019 · 5 comments
Open

Dumps String with newlines to Multiline? #236

simplesteph opened this issue Mar 9, 2019 · 5 comments
Labels
component: encoder Related to serialising in `toml.dump` syntax: strings Related to string literals type: feature A self-contained enhancement or new feature

Comments

@simplesteph
Copy link

How can I achieve this?

    print(toml.dumps({"test": "foo\n\nbar"}))

Should be

test = """
foo

bar
"""
@simplesteph simplesteph changed the title Dumps to Multiline? Dumps String with newlines to Multiline? Mar 9, 2019
@simplesteph
Copy link
Author

simplesteph commented Mar 9, 2019

I created my own wrapping encoder which is super dirty but works. I'm not going to do a PR because I'm not sure it fully respects the specs on the edge cases (especially if the enclosed string has triple quotes), but it works for my use case as I don't have triple quotes in my strings

import datetime
import re
import sys

from toml.decoder import InlineTableDict
from toml.encoder import _dump_str, _dump_float, _dump_time

if sys.version_info >= (3,):
    unicode = str

def _dump_str_new(v):
    multilines = v.split('\n')
    if len(multilines) > 1:
        return unicode('"""\n' + v.strip() + '\n"""')
    else:
        return _dump_str(v)

class MyTomlEncoder(object):

    def __init__(self, _dict=dict, preserve=False):
        self._dict = _dict
        self.preserve = preserve
        self.dump_funcs = {
            str: _dump_str_new,
            unicode: _dump_str_new,
            list: self.dump_list,
            bool: lambda v: unicode(v).lower(),
            int: lambda v: v,
            float: _dump_float,
            datetime.datetime: lambda v: v.isoformat().replace('+00:00', 'Z'),
            datetime.time: _dump_time,
            datetime.date: lambda v: v.isoformat()
        }

    def get_empty_table(self):
        return self._dict()

    def dump_list(self, v):
        retval = "["
        for u in v:
            retval += " " + unicode(self.dump_value(u)) + ","
        retval += "]"
        return retval

    def dump_inline_table(self, section):
        """Preserve inline table in its compact syntax instead of expanding
        into subsection.

        https://github.com/toml-lang/toml#user-content-inline-table
        """
        retval = ""
        if isinstance(section, dict):
            val_list = []
            for k, v in section.items():
                val = self.dump_inline_table(v)
                val_list.append(k + " = " + val)
            retval += "{ " + ", ".join(val_list) + " }\n"
            return retval
        else:
            return unicode(self.dump_value(section))

    def dump_value(self, v):
        # Lookup function corresponding to v's type
        dump_fn = self.dump_funcs.get(type(v))
        if dump_fn is None and hasattr(v, '__iter__'):
            dump_fn = self.dump_funcs[list]
        # Evaluate function (if it exists) else return v
        return dump_fn(v) if dump_fn is not None else self.dump_funcs[str](v)

    def dump_sections(self, o, sup):
        retstr = ""
        if sup != "" and sup[-1] != ".":
            sup += '.'
        retdict = self._dict()
        arraystr = ""
        for section in o:
            section = unicode(section)
            qsection = section
            if not re.match(r'^[A-Za-z0-9_-]+$', section):
                if '"' in section:
                    qsection = "'" + section + "'"
                else:
                    qsection = '"' + section + '"'
            if not isinstance(o[section], dict):
                arrayoftables = False
                if isinstance(o[section], list):
                    for a in o[section]:
                        if isinstance(a, dict):
                            arrayoftables = True
                if arrayoftables:
                    for a in o[section]:
                        arraytabstr = "\n"
                        arraystr += "[[" + sup + qsection + "]]\n"
                        s, d = self.dump_sections(a, sup + qsection)
                        if s:
                            if s[0] == "[":
                                arraytabstr += s
                            else:
                                arraystr += s
                        while d:
                            newd = self._dict()
                            for dsec in d:
                                s1, d1 = self.dump_sections(d[dsec], sup +
                                                            qsection + "." +
                                                            dsec)
                                if s1:
                                    arraytabstr += ("[" + sup + qsection +
                                                    "." + dsec + "]\n")
                                    arraytabstr += s1
                                for s1 in d1:
                                    newd[dsec + "." + s1] = d1[s1]
                            d = newd
                        arraystr += arraytabstr
                else:
                    if o[section] is not None:
                        retstr += (qsection + " = " +
                                   unicode(self.dump_value(o[section])) + '\n')
            elif self.preserve and isinstance(o[section], InlineTableDict):
                retstr += (qsection + " = " +
                           self.dump_inline_table(o[section]))
            else:
                retdict[qsection] = o[section]
        retstr += arraystr
        return (retstr, retdict)

and to use

            text = toml.dumps(data, MyTomlEncoder())

@retorquere
Copy link

When I use this, I get

  File "/usr/local/lib/python3.7/site-packages/toml/encoder.py", line 47, in dumps
    addtoretval, sections = encoder.dump_sections(o, "")
TypeError: dump_sections() missing 1 required positional argument: 'sup'

@vvasuki
Copy link

vvasuki commented Apr 6, 2021

My minimal implementation https://github.com/sanskrit-coders/sanskrit_data/blob/67e0999be6f8bf7fff761f0484141e03b9e551f4/sanskrit_data/toml_helper.py#L6

@EternityForest
Copy link

@vvasuki could we add a replace('"""','\"""') as a minimal way to correctly handle escaping any embedded triple quotes?

@vvasuki
Copy link

vvasuki commented May 13, 2021

@vvasuki could we add a replace('"""','"""') as a minimal way to correctly handle escaping any embedded triple quotes?

Done (edited link in prior message.)

@pradyunsg pradyunsg added component: encoder Related to serialising in `toml.dump` type: feature A self-contained enhancement or new feature syntax: strings Related to string literals labels Apr 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: encoder Related to serialising in `toml.dump` syntax: strings Related to string literals type: feature A self-contained enhancement or new feature
Projects
None yet
Development

No branches or pull requests

5 participants