Add ByteString IsUTF8

[f-lapinski]

Change Log

Added

• ByteString IsUTF8 function with Test

Fixed

• Add Missing Test for return type of function

Changed

Removed

Deprecated

Security

---

Description

Add the IsUtf8() symfony function

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+------------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev           |
+-----------------------+-------------------+------+-----+-----------------+------------------+------------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.799mb +0.01%  | 551.668ms -0.86% | ±2.74% +1212.26% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.872mb +0.01%  | 1.070s +0.88%    | ±0.73% +318.99%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.315mb +0.00% | 918.962ms +3.00% | ±0.90% +280.91%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.529mb +0.02%  | 35.554ms +0.33%  | ±0.23% -79.35%   |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.504mb +0.02%  | 601.122ms -0.30% | ±0.83% +118.77%  |
+-----------------------+-------------------+------+-----+-----------------+------------------+------------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 127.325mb +0.00% | 75.259ms +5.60% | ±0.86% -11.18% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 63.998mb +0.00%  | 106.477ms +2.56% | ±0.48% +40.95%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 84.345mb +0.00%  | 100.203ms +1.88% | ±0.23% -67.06%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 161.186mb +0.00% | 21.291s +3.36%   | ±0.54% +124.04% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.995mb +0.00%  | 32.106ms +1.01%  | ±0.83% +283.72% |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+
| benchmark         | subject                    | revs | its | mem_peak         | mode             | rstdev                        |
+-------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 105.968mb +0.00% | 464.898ms +1.36% | ±2.15% -29.02%                |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.159mb +0.00%  | 240.711ms +4.78% | ±2.00% +188.37%               |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.681mb +0.00%  | 52.371ms +4.85%  | ±0.58% -46.84%                |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 43.804mb +0.00%  | 368.662ms +1.45% | ±1.42% +60.80%                |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 11.613mb +0.01%  | 74.346ms +1.31%  | ±0.25% -68.72%                |
| RowsBench         | bench_chunk_10_on_10k      | 2    | 3   | 97.012mb +0.00%  | 5.363ms +71.27%  | ±1.93% +581.18%               |
| RowsBench         | bench_diff_left_1k_on_10k  | 2    | 3   | 114.295mb +0.00% | 191.560ms +4.65% | ±1.70% +611.02%               |
| RowsBench         | bench_diff_right_1k_on_10k | 2    | 3   | 97.015mb +0.00%  | 19.080ms +2.37%  | ±1.74% +1.43%                 |
| RowsBench         | bench_drop_1k_on_10k       | 2    | 3   | 97.887mb +0.00%  | 2.340ms +47.80%  | ±0.74% -54.38%                |
| RowsBench         | bench_drop_right_1k_on_10k | 2    | 3   | 97.887mb +0.00%  | 2.388ms +63.52%  | ±2.35% -38.16%                |
| RowsBench         | bench_entries_on_10k       | 2    | 3   | 96.047mb +0.00%  | 6.549ms +52.18%  | ±0.72% -47.50%                |
| RowsBench         | bench_filter_on_10k        | 2    | 3   | 96.576mb +0.00%  | 17.728ms +7.38%  | ±0.30% +84.73%                |
| RowsBench         | bench_find_on_10k          | 2    | 3   | 96.576mb +0.00%  | 17.802ms +9.21%  | ±0.88% +44.00%                |
| RowsBench         | bench_find_one_on_10k      | 10   | 3   | 95.268mb +0.00%  | 2.006μs +5.58%   | ±2.32% +19838029799670000.00% |
| RowsBench         | bench_first_on_10k         | 10   | 3   | 95.268mb +0.00%  | 0.500μs +25.00%  | ±0.00% -100.00%               |
| RowsBench         | bench_flat_map_on_1k       | 2    | 3   | 104.486mb +0.00% | 16.755ms +15.43% | ±2.83% +160.83%               |
| RowsBench         | bench_map_on_10k           | 2    | 3   | 134.553mb +0.00% | 78.994ms +8.04%  | ±0.49% -22.53%                |
| RowsBench         | bench_merge_1k_on_10k      | 2    | 3   | 97.096mb +0.00%  | 2.333ms +76.36%  | ±0.81% -66.34%                |
| RowsBench         | bench_partition_by_on_10k  | 2    | 3   | 100.394mb +0.00% | 69.382ms +7.92%  | ±0.60% -59.13%                |
| RowsBench         | bench_remove_on_10k        | 2    | 3   | 98.149mb +0.00%  | 5.181ms +29.68%  | ±3.28% +54.63%                |
| RowsBench         | bench_sort_asc_on_1k       | 2    | 3   | 95.557mb +0.00%  | 46.206ms +12.37% | ±1.06% +87.65%                |
| RowsBench         | bench_sort_by_on_1k        | 2    | 3   | 95.557mb +0.00%  | 44.773ms +7.09%  | ±1.22% -9.97%                 |
| RowsBench         | bench_sort_desc_on_1k      | 2    | 3   | 95.557mb +0.00%  | 45.015ms +8.90%  | ±1.47% +2.76%                 |
| RowsBench         | bench_sort_entries_on_1k   | 2    | 3   | 97.708mb +0.00%  | 9.472ms +12.38%  | ±2.63% +288.50%               |
| RowsBench         | bench_sort_on_1k           | 2    | 3   | 95.458mb +0.00%  | 31.869ms +7.43%  | ±1.54% +142.00%               |
| RowsBench         | bench_take_1k_on_10k       | 10   | 3   | 95.268mb +0.00%  | 16.510μs +12.75% | ±2.68% +20.02%                |
| RowsBench         | bench_take_right_1k_on_10k | 10   | 3   | 95.268mb +0.00%  | 19.100μs +25.98% | ±0.43% -86.00%                |
| RowsBench         | bench_unique_on_1k         | 2    | 3   | 114.296mb +0.00% | 191.646ms +1.97% | ±0.40% -63.34%                |
+-------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+

[codecov]

Codecov Report

Attention: Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.

Project coverage is 83.05%. Comparing base (dbba71d) to head (3c80053).
Report is 5 commits behind head on 1.x.

Additional details and impacted files

@@            Coverage Diff             @@
##              1.x    #1470      +/-   ##
==========================================
- Coverage   83.05%   83.05%   -0.01%     
==========================================
  Files         663      664       +1     
  Lines       17853    17864      +11     
==========================================
+ Hits        14828    14837       +9     
- Misses       3025     3027       +2     

Components	Coverage Δ
etl	85.78% <90.90%> (+<0.01%)	⬆️
cli	86.73% <ø> (ø)
lib-array-dot	94.53% <ø> (ø)
lib-azure-sdk	62.56% <ø> (ø)
lib-doctrine-dbal-bulk	97.36% <ø> (ø)
lib-filesystem	76.75% <ø> (ø)
lib-parquet	84.33% <ø> (ø)
lib-parquet-viewer	82.02% <ø> (ø)
lib-rdsl	87.09% <ø> (ø)
lib-snappy	91.16% <ø> (-0.47%)	⬇️
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	96.38% <ø> (ø)
symfony-http-foundation	77.10% <ø> (ø)
adapter-chartjs	86.45% <ø> (ø)
adapter-csv	89.57% <ø> (ø)
adapter-doctrine	88.68% <ø> (ø)
adapter-elasticsearch	97.19% <ø> (ø)
adapter-google-sheet	78.04% <ø> (ø)
adapter-http	59.15% <ø> (ø)
adapter-json	90.90% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.75% <ø> (ø)
adapter-parquet	80.85% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	83.15% <ø> (ø)

[norberttech]

Nice one @f-lapinski, you are making an excellent progress, you should be proud of yourself, I'm proud of you! 🍻