Duplicate key error while adding two mentions which are same

[saikalyan9981]

Suppose that I have two mentions (say for example zip-code and tax code) whose matchers return true (checking 5 digit regex match for both mentions) for the same span in document, then I think Fonduer is throwing this error. please help me in resolving this.

sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "context_stable_id_key"
DETAIL:  Key (stable_id)=(1443208965_10_subset::span_mention:23313:23321) already exists.

[SQL: INSERT INTO context (type, stable_id) VALUES (%(type)s, %(stable_id)s) RETURNING context.id]

[saikalyan9981]

Note that this was happening because I was using two MentionExtractors for the two mentions.

[HiromuHota]

What would happen if you use only one MentionExtractor for the two mentions?

[saikalyan9981]

I need two mention Extractors in my program because first I'm extracting some candidates with certain matchers and labelling functions, and based on those candidates extracted, I need to extract some other candidates, in next round.
Do you think it's bad to have such a pipeline?

[HiromuHota]

I'm not saying this is a bad idea or not. I'm just trying to understand the issue.
From your response, seems that you've never tried to extract these two mentions with a single MentionExtractor.

Which version of Fonduer are you using?

[saikalyan9981]

I'm using "0.8.2". I'm using a singleMentionExtractor for 4 mentions in the first run. I need to have those mentions extracted to extract other mentions in my second run, so using a second mention extractor. If I use a single extractor for all mentions, it's giving no errors for the same context match. If I use two extractors for 2 mentions, its throwing above error for the same context match.

[HiromuHota]

If I use a single extractor for all mentions, it's giving no errors for the same context match.

@saikalyan9981 Thanks for this valuable information! I'll look into this.
Meanwhile, you could use a single extractor as a workaround if the second extraction does not depend on the first one.

[HiromuHota]

I found that this issue is a regression caused by #368. A workaround would be to set `clear=False` like `mention_extractor.apply(clear=False, ...)`. @saikalyan9981 can you try this workaround? Please let me know how it works.

[HiromuHota]

I think this issue is similar (not identical) to #424.
You will have an error even when you set clear=False.
A workaround would be to downgrade to v0.7.1 or to upgrade to v0.8.3, which is not released yet.

[HiromuHota]

Closing as v0.8.3 has been released.