Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DOC]: CLX/Morpheus Parity Comparison - Engineering #508

Closed
8 tasks done
BartleyR opened this issue Nov 29, 2022 · 3 comments
Closed
8 tasks done

[DOC]: CLX/Morpheus Parity Comparison - Engineering #508

BartleyR opened this issue Nov 29, 2022 · 3 comments
Assignees
@efajardo-nv
Labels
doc Improvements or additions to documentation

Comments

@BartleyR
Copy link
Contributor

BartleyR commented Nov 29, 2022 •
edited by efajardo-nv
Loading

How would you describe the priority of this documentation request

Medium

Describe the future/missing documentation

The current RAPIDS CLX library has functionality that could be integrated into Morpheus. As the first step towards this, we wish to understand the parity between CLX and Morpheus in terms of the APIs, capabilities, developer tools, and methods that exist in both.

Where have you looked?

No such documentation exist in CLX or Morpheus

Tasks

Generated from @efajardo-nv's comment below

Tasks
Preview Give feedback

Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I have searched the open documentation issues and have found no duplicates for this bug report
@BartleyR BartleyR added the doc Improvements or additions to documentation label Nov 29, 2022
@BartleyR BartleyR moved this to Todo in Morpheus Boards Nov 29, 2022
@efajardo-nv
Copy link
Contributor

Possible CLX functionality to migrate:

LODA anomaly detection

Multi-class Sequence classifier

  • clx.analytics.multiclass_sequence_classifier.MulticlassSequenceClassifier
  • Can be used for various classifier use cases such as Root Cause Analysis which currently uses binary sequence classifier.

Splunk Notable Parser

Windows Event Log Parser

  • clx.parsers.windows_event_parser.WindowsEventParser

Zeek Parser

Domain Generation Algorithm (DGA) Detection

DNS Extractor

  • clx.dns.dns_extractor

@efajardo-nv efajardo-nv mentioned this issue Apr 19, 2023
Merged
@efajardo-nv
Copy link
Contributor

CLX LODA migrated via nv-morpheus/morpheus-experimental#38

@efajardo-nv efajardo-nv mentioned this issue May 2, 2023
Merged
rapids-bot bot pushed a commit that referenced this issue May 5, 2023
@efajardo-nv
Migrate CLX parsers (#894)
0f53830 
- Splunk Notable parser
- Windows Event parser
- URL parser
- Zeek parser
- IP parser
- Unit tests

Tasks from Issue #508

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #894
@efajardo-nv efajardo-nv mentioned this issue May 18, 2023
Merged
rapids-bot bot pushed a commit that referenced this issue May 31, 2023
@efajardo-nv
Multi-class Sequence Classifier (#952)
918f02f 
- Migrate CLX multi-class sequence classifier. Update existing binary classifier to support more than two labels.
- Update existing phishing and root cause models to use models generated from updated sequence classifier.
- Remove duplicated Python files from `training-tuning-scripts`. Add single version of these files to `common` directory and symlinks to `common` for phishing and root cause training.
- Remove duplicated tokenizer hash files. Update training scripts to point to hash file in `morpheus/data`.
- Update phishing and root cause training/inference scripts and notebooks.

Task from issue #508

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #952
rapids-bot bot pushed a commit to nv-morpheus/morpheus-experimental that referenced this issue Jun 5, 2023
@efajardo-nv
Migrate CLX DGA Detection (#46)
f59aada 
- Migrate code, notebook, dataset for CLX DGA Detection
- Remove references to CLX
- Example Morpheus pipeline
- Triton model repo with ONNX model
- Fix bug in existing code caused by upstream cudf updates
- Add new training and validation data
- Merge version updates from PR #49
- Remove `cuml` dependency so scripts/notebooks can also be run in Morpheus container 

Task from nv-morpheus/Morpheus#508

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)
  - https://github.com/raykallen

URL: #46
@efajardo-nv
Copy link
Contributor

all tasks completed/merged.

@github-project-automation github-project-automation bot moved this from Todo to Done in Morpheus Boards Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@efajardo-nv efajardo-nv

Labels
doc Improvements or additions to documentation
Projects
Morpheus Boards
Status: Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@BartleyR @efajardo-nv @jarmak-nv