Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/add analytics #387

Merged
merged 2 commits into from
Jun 1, 2023
Merged

Feature/add analytics #387

merged 2 commits into from
Jun 1, 2023

Conversation

alallema
Copy link
Contributor

@alallema alallema commented May 30, 2023

Following this central issue meilisearch/integration-guides#150
This repo now add a user-agent in the header of docs-scraper

@alallema alallema added the enhancement New feature or request label May 30, 2023
@alallema alallema linked an issue May 30, 2023 that may be closed by this pull request
4 tasks
@alallema alallema force-pushed the feature/add-analytics branch from e18ab2b to 5f2b3a3 Compare May 30, 2023 11:33
Copy link
Member

@brunoocasali brunoocasali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it is not ready yet, but this is the correct header name! ;)

scraper/src/config/version.py Outdated Show resolved Hide resolved
@alallema alallema force-pushed the feature/add-analytics branch from 137aea6 to 96cbb42 Compare May 30, 2023 13:47
@alallema alallema marked this pull request as ready for review May 30, 2023 13:47
@alallema alallema force-pushed the feature/add-analytics branch from 96cbb42 to eb54b77 Compare May 30, 2023 13:55
@alallema alallema force-pushed the feature/add-analytics branch from eb54b77 to 72daf28 Compare May 30, 2023 14:04
Copy link
Member

@brunoocasali brunoocasali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alallema the user agent implemented will be used to inform the scrapped pages that they are going to be scraped by Meilisearch.

But the one that we need is this one: meilisearch/integration-guides#150 which is the user-agent we send to meilisearch.

So I assume you had to pass it from Meilisearch Python.

scraper/src/meilisearch_helper.py Outdated Show resolved Hide resolved
@alallema alallema force-pushed the feature/add-analytics branch 2 times, most recently from 044c385 to fc89a6e Compare May 31, 2023 15:43
meili-bors bot added a commit to meilisearch/meilisearch-python that referenced this pull request Jun 1, 2023
771: Adding a header parameters to the client constructor r=alallema a=alallema

To enable and facilitate the integration of a user-agent into all SDKs. [see related issue](meilisearch/integration-guides#150) and specifically for [the docs-scraper integration](meilisearch/docs-scraper#387) a new parameter client_agent is now available in the client constructor.



Co-authored-by: alallema <[email protected]>
Co-authored-by: Amélie <[email protected]>
Co-authored-by: Paul Sanders <[email protected]>
@alallema alallema requested a review from brunoocasali June 1, 2023 08:48
@alallema alallema force-pushed the feature/add-analytics branch 2 times, most recently from 86351c2 to e5d627c Compare June 1, 2023 09:29
Copy link
Contributor

@bidoubiwa bidoubiwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good work ✨

scraper/src/config/version.py Show resolved Hide resolved
scraper/src/config/version.py Outdated Show resolved Hide resolved
tests/config_loader/basic_test.py Outdated Show resolved Hide resolved
tests/meilisearch_helper/version_test.py Show resolved Hide resolved
)

# Then
assert actual.meilisearch_client.http.headers['User-Agent'] == f"Meilisearch Python (v0.27.0);Meilisearch DocsScraper (v{__version__})"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that all these headers are not separated by a white space makes matching harder on our analytics tools no?
It might not be a prefix search strategy though!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question, I have no idea, but I think @brunoocasali will be better able to answer, as these entries will be used through Amplitude.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope @bidoubiwa it is correct like this, they use the ; semicolon as the separator.

More about it here in this comment: meilisearch/integration-guides#150 (comment)

@alallema alallema force-pushed the feature/add-analytics branch from e5d627c to b7b1931 Compare June 1, 2023 10:12
@alallema alallema requested a review from bidoubiwa June 1, 2023 13:07
Copy link
Contributor

@bidoubiwa bidoubiwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🔥🔥

@alallema alallema merged commit 514acf4 into main Jun 1, 2023
@alallema alallema deleted the feature/add-analytics branch June 1, 2023 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add analytics to repo
3 participants