Create json schema for yaml configuration

[lecaros]

Is your feature request related to a problem? Please describe.
IDEs can use JSON schemas to help users define their YAML configurations through completion.

Describe the solution you'd like
To have a JSON schema available either through public repo or published in https://www.schemastore.org.

[niedbalski]

@patrick-stephens please lets add the fluent-bit -J output to the nightly/unstable + released artefacts.

[patrick-stephens]

In progress, should be ready once the automation is running correctly.

[patrick-stephens]

This should be present in the release jobs now. It'll be in the S3 bucket @niedbalski but we could also copy to the packages server, although I'd say let's align with #5098 and stop duplicating things.

Re-open if you want it elsewhere.

[patrick-stephens]

Manually deployed here:

• https://releases.fluentbit.io/1.9/fluent-bit-schema.json
• https://releases.fluentbit.io/1.9/fluent-bit-schema-pretty.json

Minor niggle in the S3 sync should be resolved by #5270

[ffflabs]

Manually deployed here:

• https://releases.fluentbit.io/1.9/fluent-bit-schema.json

@patrick-stephens I used that schema to infer a json-schema compliant with draft 7. I believe schemastore would ask for that format.

https://calyptia-vite.ffflabs.com/fluentbit-jsonschema.json

[patrick-stephens]

@ffflabs just checking that is a good thing right or do we need to do something else? Sorry, I'm not really an expert on schema store so just checking if there is more stuff that needs doing.

[henrysachs]

i couldn't find schemas for the latest releases. Did I miss something or did this get lost?

[patrick-stephens]

They should be around somewhere but it has been a while - they're just generated from the build anyway via the -J parameter, e.g. docker run --rm -it fluent/fluent-bit -J

[patrick-stephens]

It is being generated:

fluent-bit/.github/workflows/call-build-images.yaml

Line 179 in 5686334

docker run --rm -t ${{ inputs.registry }}/${{ inputs.image }}:${{ inputs.version }} -J > fluent-bit-schema-${{ inputs.version }}.json

We then push it to the bucket:

fluent-bit/.github/workflows/staging-build.yaml

Line 111 in 5686334

aws s3 sync "artifacts/" "s3://${AWS_S3_BUCKET}/${DEST_DIR}" --no-progress

That does ignore errors - older builds cannot support it - but it was sync'd: https://github.com/fluent/fluent-bit/actions/runs/5652281163/job/15354447347#step:4:11

I cannot see any specific filtering of it:

fluent-bit/.github/workflows/staging-release.yaml

Line 388 in 5686334

aws s3 sync release/ "s3://${{ secrets.AWS_S3_BUCKET_RELEASE_SOURCES }}" --delete --follow-symlinks --no-progress

My guess is some of the refactoring we have had to do elsewhere may have affected it.

[patrick-stephens]

Ah, it is there: https://packages.fluentbit.io/2.1.8/
That's not the releases subdomain so that seems to be the change.
Please submit a PR to update the docs.

[henrysachs]

Okay thanks a lot for digging into this!

[henrysachs]

I'm not that deep into json schemas, but this version seems like a version of json schema other tools won't understand. How is this schema generated?

[patrick-stephens]

I would raise a separate issue for problems with it @henrysachs. I don't really know how it is generated other than we invoke the -J option. No one else will see these comments so asking it here is probably not best!

[henrysachs]

okay owm :)

[JustinGrote]

I generated a schema here since @ffflabs is 404'd
#9289 (comment)