Numeric fields are not of numeric type in resulting JSON

[kluen]

I'm trying to produce numeric fields in the JSON output when using consoleJson.

Here is an example where I am trying to print the line number of the trace:

//> using scala "2.13"
//> using lib "dev.zio::zio:2.0.0"
//> using lib "dev.zio::zio-logging:2.1.0"

import zio._
import zio.logging._

object Example extends ZIOAppDefault {

  val run = ZIO.log("Hey! Listen!")

  override val bootstrap: ZLayer[ZIOAppArgs with Scope, Any, Any] =
    Runtime.removeDefaultLoggers >>> consoleJson(format)

  lazy val format = LogFormat.label("theLine", traceLine)

  lazy val traceLine = LogFormat.make {
    (builder, trace, fiberId, _, _, _, _, _, _) =>
      trace match {
        case Trace(_, _, line) => builder.appendNumeric(line)
      }
  }
}

which results in

{"theLine":"10"}

Note that the "10" is actually a string, not a number.

Looking at the code, I was wondering how exactly LogAppender.appendNumeric is intended to be used anyway. It seems odd that it excepts any type: def appendNumeric[A](numeric: A): Unit.

[justcoon]

hi @kluen, thank you for report

current implementation of json LogAppender converting number to string

def appendNumeric[A](numeric: A): Unit = appendText(numeric.toString)

not sure what was reason for that, but i will discuss it, and let you know (i am not author of this LogAppender)

in general it is about how specific LogAppender is implemented, at the end

but also, I think, if there is possibility to have it like number, it should be number, not string ...

[justcoon]

hi @davidlar,

as you are author of json LogAppender, can you say something about this problem? Is there some reason why numeric values are converted to string, and not handled like numbers?

Thank you very much.

[davidlar]

Hi!

As I remember appendNumeric wasn't used by any existing log format, so I didn't bother. It should be easy to fix though.
Is there any need for it?

[justcoon]

hi @davidlar,

i think, in general most of log appenders transforming log inputs to string, so maybe that is reason why it was like that

but if there is possibility to use specific data types, in my opinion, we should use it, so i think we should have it also in json like number not like string

[kluen]

Is there any need for it?

@davidlar In my particular case, I need some fields to be numbers so that the log format adheres to a common schema defined by our central log aggregation (we are trying to mimic the output of the widely used logback-logstash-encoder). I believe log aggregation with a schema like this might be a fairly common use case.

[davidlar]

I think we can fix it for simple cases like

lazy val format = LogFormat.label("theLine", traceLine)

lazy val traceLine = LogFormat.make {
  (builder, trace, fiberId, _, _, _, _, _, _) =>
    trace match {
      case Trace(_, _, line) => builder.appendNumeric(line)
    }
}

but if you concat formats like this

lazy val format = LogFormat.label("theLine", enclosingClass |-| traceLine)

it will result in a json string, of course.

Does that feel ok to you?

[justcoon]

Hi @davidlar, i think your proposal is good. Are you willing to make these changes? Thanks.

[davidlar]

Sure!

[justcoon]

hi @kluen, new version was released, (thanks to @davidlar for fix), can you let us know, if it is ok now? Thank you very much.

[kluen]

I just checked with my example above and zio-logging 2.1.1. It works as expected.

Thanks @davidlar for the fix and @justcoon for driving the topic forward!