Firecrawl URL Importing is throwing error

[ukiras123]

Description

Installation

• pip install goldenverba
• pip install from source
• Docker installation

If you installed via pip, please specify the version:

Weaviate Deployment

• Local Deployment
• Docker Deployment
• Cloud Deployment

Configuration

Reader:
Chunker:
Embedder:
Retriever:
Generator:

Steps to Reproduce

Try to import using Firecrawl URL. It is throwing error:
ℹ FileStatus.ERROR | 2024-09-10T04:36:25.979Z | Import for New
Firecrawl Job failed: Reader Firecrawl failed with: 1 validation error for
FileConfig metadata Field required [type=missing, input_value={'fileID':
'2024-09-10T04... settings', 'took': 0}}}, input_type=dict] For further
information visit https://errors.pydantic.dev/2.9/v/missing | 0

Additional context

It is not working with any website I tried. Firecrawl has also updated its version so we might want to do that as well.

[ukiras123]

ℹ FileStatus.ERROR | 2024-09-10T04:36:25.979Z | Import for New
Firecrawl Job failed: Reader Firecrawl failed with: 1 validation error for
FileConfig metadata Field required [type=missing, input_value={'fileID':
'2024-09-10T04... settings', 'took': 0}}}, input_type=dict] For further
information visit https://errors.pydantic.dev/2.9/v/missing | 0

[thomashacker]

Thanks for the issue! We'll look into it

[narmaku]

Same issue while trying to import github repo that contains python code.

[thomashacker]

Thanks a lot, there was a bug in the Firecrawl Reader code, should be fixed in the upcoming version 2.1

[thomashacker]

Version 2.1 got released, let me know if the error still persists and feel free to reopen