Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Processor: Datetime from String #1865

Closed
bossenti opened this issue Sep 4, 2023 · 2 comments · Fixed by #2207
Closed

New Processor: Datetime from String #1865

bossenti opened this issue Sep 4, 2023 · 2 comments · Fixed by #2207
Assignees
Labels
enhancement New feature or request good first issue Good for newcomers hacktoberfest Indicates an issue for hacktoberfest event 2023. pipeline elements Relates to pipeline elements
Milestone

Comments

@bossenti
Copy link
Contributor

bossenti commented Sep 4, 2023

Description

A user should have the ability parse datetime values from string values in a pipeline. Therefore, a new processor is required that provides the functionality to convert a string value to a datetime value based on a given string format.

Example Configuration

DatetimeFormat : A format string that describes the format of the datetime string

The datetime format should be configurable with a conveniant UI dialog, ideally similar to the our Notification sink:
image

Additional Functionality

If you have any other ideas for the functionality of this new processor, please feel free to add them here.

Mentoring

As this issue is marked as good first issue: one of @dominikriemer, @tenthe, or @bossenti are happy to provide help for getting started, just tag (one of) them if you want to start working on this issue and need some help.
This issue is ideal for people that are interested in getting started with StreamPipes and its SDK.

StreamPipes Committer

I acknowledge that I am a maintainer/committer of the Apache StreamPipes project.

@bossenti bossenti added enhancement New feature or request good first issue Good for newcomers pipeline elements Relates to pipeline elements labels Sep 4, 2023
@bossenti bossenti added this to the 1.0.0 milestone Sep 4, 2023
@bossenti bossenti added the hacktoberfest Indicates an issue for hacktoberfest event 2023. label Sep 7, 2023
@dshunter107
Copy link
Contributor

Hello @tenthe , I am interested in working on this problem, but I am quite new to the GitHub community. Would you be able to point me in the direction of which files/packages would be of interest to me. I'm still getting used to wading through large projects.

Also, the description makes this seem like all I would need to do is implement some simple function like a LocalDate.parse("2023-18-05"); . Is this understanding correct, or would it need to be more involved.

@bossenti
Copy link
Contributor Author

bossenti commented Sep 9, 2023

Hey @dshunter107,

thanks for your interest in contributing here 🙏🏼
Don't worry, we are happy to assist you and guide you through your first steps on GitHub.

One important thing is the git flow for open source projects: https://www.asmeurer.com/git-workflow/
Please read through this to know how you can open a PR in our repository.

Since this issue involves introducing a new pipeline element, the following tutorial might be relevant for you: https://streampipes.apache.org/docs/extend-tutorial-data-processors
There might be some parts that are not completely up to date, we already have this on our radar. But if you face any issues, we are ready to help 🙂

Since you asked, where to place the new processing element in our repo, I'd recommend you choose streampipes-processors-transformation-jvm: https://github.com/apache/streampipes/blob/eccb29ff437ea5c560d16bbf86e95e7f1011502f/streampipes-extensions/streampipes-processors-transformation-jvm/src/main/java/org/apache/streampipes/processors/transformation/jvm/processor

Yes, that's probably the only thing we need 🙂 When writing this issue I thought way too complex, so I really liked your proposal.
The only aspect we need to pay attention to is the target time zone. Your snippet would just use the timezone of the JVM where the processor is running, right? This might not always be the appropriate one

dshunter107 added a commit to dshunter107/streampipes that referenced this issue Nov 20, 2023
Closes apache#1865

This processor parses a DateTime string. It converts the string into a
ZonedDateTime variable, and places the variable in the event variable.
Tests are made to ensure that the code works properly.

I am still somewhat new, so any feedback is welcome.
dominikriemer pushed a commit that referenced this issue Dec 8, 2023
* Creation of DateTimeProcessor

Closes #1865

This processor parses a DateTime string. It converts the string into a
ZonedDateTime variable, and places the variable in the event variable.
Tests are made to ensure that the code works properly.

I am still somewhat new, so any feedback is welcome.

* Updates to parse datetime.

Incorporates feedback such as the following:
* Way to initialize variables with constructors
* simplifies methods with more efficient methods
* creates the resource file for the processor
* standardizes the names of inputs (i.e field)
* creates the documentation for the processor

* I had renamed the processor, but I forgot to rename the Testing file.

* improve assets

* improve assets

* adapt parameter name in test

* Registering Processing Element and Aligning Ids

* Drop down instead of radio group.

Also change inputTimeZone description in resource files

* improve parameter description

* put timestamp in event

* Temporary workaround

* Includes output strategy for time zone.

One possible workaround is to use the time zone to reconstitute the
datetime after the event has been collected.

* minor final improvements

* style: adapt to checkstyle

---------

Co-authored-by: bossenti <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers hacktoberfest Indicates an issue for hacktoberfest event 2023. pipeline elements Relates to pipeline elements
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants