regression test artifacts

SophistSolutions · Mar 1, 2025 · 108a115 · 108a115
1 parent 4e48d8c
commit 108a115
Show file tree

Hide file tree

Showing 15 changed files with 28,788 additions and 2 deletions.
diff --git a/Release-Notes.md b/Release-Notes.md
@@ -104,14 +104,14 @@ especially those they need to be aware of when upgrading.
 
 - Compilers Tested/Supported
   - g++ { 11, 12, 13, 14 }
-  - Clang++ { unix: 15, 16, 17, 18, 19; XCode: 15.2, 15.3, 16.0}
+  - Clang++ { unix: 15, 16, 17, 18, 19; XCode: 15.2, 15.3, 16.0 }
   - MSVC: { 17.13.2 }
 - OS/Platforms Tested/Supported
   - Windows
     - Windows 11 version 24H2
     - mcr.microsoft.com/windows/servercore:ltsc2022 (build/run under docker)
       - cygwin (latest as of build-time from CHOCO)
-      - MSYS (msys2-base-x86_64-20230127.sfx.exe)
+      - MSYS (msys2-base-x86_64-20241208.sfx.exe)
     - WSL v2
   - MacOS
     - 15.0.1 - arm64/m1 chip
@@ -129,6 +129,8 @@ especially those they need to be aware of when upgrading.
 - Known (minor) issues with regression test output
   - raspberrypi
     - 'badssl.com site failed with fFailConnectionIfSSLCertificateInvalid = false: SSL peer certificate or SSH remote key was not OK (havent investigated but seems minor)
+  - windows/cygwin
+    - SSDPClient issues have todo with quirks of windows networking - reboot PC and fine...
 
 ---
 

diff --git a/...storicalPerformanceRegressionTestResults/3.0/PerformanceDump-WSL_Ubuntu_Simple-3.0d16.txt b/...storicalPerformanceRegressionTestResults/3.0/PerformanceDump-WSL_Ubuntu_Simple-3.0d16.txt
@@ -0,0 +1,194 @@
+Performance score 1.0 means both sides equal (ratio), and tests setup so lower is generally better
+
+[[[Started testing at: Fri Feb 28 16:10:49 2025]]]
+
+Using TIME MULTIPLIER: 15
+
+Test Test of simple locking strategies (mutex v shared_ptr copy) (mutex vs shared_ptr<> copy)
+	PERFORMANCE_SCORE	0.71
+	DETAILS:         	[baseline test 3.08991 secs, and comparison 2.17903 sec, and warnIfPerfScore > 0.9, and perfScore=0.705207]
+	                 	shared_ptr<> copy is FASTER
+
+Test Test of simple locking strategies (mutex v SpinLock) (mutex vs SpinLock)
+	PERFORMANCE_SCORE	1
+	DETAILS:         	[baseline test 3.03926 secs, and comparison 3.15955 sec, and warnIfPerfScore > 0.51, and perfScore=1.03958]
+	                 	SpinLock is ***SLOWER***
+
+Test Simple Struct With Strings Filling And Copying (wstring vs Characters::String)
+	PERFORMANCE_SCORE	0.27
+	DETAILS:         	[baseline test 3.00811 secs, and comparison 0.822664 sec, and warnIfPerfScore > 0.49, and perfScore=0.273482]
+	                 	Characters::String is FASTER
+
+Test Simple Struct With Strings Filling And Copying2 (wstring vs Characters::String)
+	PERFORMANCE_SCORE	0.22
+	DETAILS:         	[baseline test 4.69514 secs, and comparison 1.03117 sec, and warnIfPerfScore > 0.45, and perfScore=0.219626]
+	                 	Characters::String is FASTER
+
+Test Simple String append test (+='string object') 10x (wstring vs Characters::String)
+	PERFORMANCE_SCORE	1.7
+	DETAILS:         	[baseline test 4.47979 secs, and comparison 7.63752 sec, and warnIfPerfScore > 4.9, and perfScore=1.70488]
+	                 	Characters::String is ***SLOWER***
+
+Test Simple String append test (+=wchar_t[]) 10x (wstring vs Characters::String)
+	PERFORMANCE_SCORE	5.6
+	DETAILS:         	[baseline test 5.34147 secs, and comparison 29.8068 sec, and warnIfPerfScore > 4.1, and perfScore=5.58026]
+	                 	Characters::String is ***SLOWER***
+
+Test Simple String append test (+=wchar_t[]) 100x (wstring vs Characters::String)
+	PERFORMANCE_SCORE	8.4
+	DETAILS:         	[baseline test 29.1284 secs, and comparison 245.058 sec, and warnIfPerfScore > 78, and perfScore=8.41303]
+	                 	Characters::String is ***SLOWER***
+
+Test String a + b (wstring vs String)
+	PERFORMANCE_SCORE	1.2
+	DETAILS:         	[baseline test 3.25104 secs, and comparison 3.97764 sec, and warnIfPerfScore > 2.1, and perfScore=1.2235]
+	                 	String is ***SLOWER***
+
+Test wstringstream << test (wstring vs Characters::String)
+	PERFORMANCE_SCORE	1.2
+	DETAILS:         	[baseline test 5.73947 secs, and comparison 7.11579 sec, and warnIfPerfScore > 1.4, and perfScore=1.2398]
+	                 	Characters::String is ***SLOWER***
+
+Test String::substr() (wstring vs Characters::String)
+	PERFORMANCE_SCORE	1.6
+	DETAILS:         	[baseline test 3.14093 secs, and comparison 5.13408 sec, and warnIfPerfScore > 1.7, and perfScore=1.63457]
+	                 	Characters::String is ***SLOWER***
+
+Test wstringstream versus BasicTextOutputStream (wstringstream vs MemoryStream<Characters::Character>)
+	PERFORMANCE_SCORE	1.2
+	DETAILS:         	[baseline test 3.73016 secs, and comparison 4.47752 sec, and warnIfPerfScore > 1.2, and perfScore=1.20035]
+	                 	MemoryStream<Characters::Character> is ***SLOWER***
+
+Test wstringstream versus StringBuilder (wstringstream vs StringBuilder)
+	PERFORMANCE_SCORE	1.1
+	DETAILS:         	[baseline test 3.45725 secs, and comparison 3.84511 sec, and warnIfPerfScore > 0.48, and perfScore=1.11219]
+	                 	StringBuilder is ***SLOWER***
+
+Test Sequence<int> basics (vector<int> vs Sequence<int>)
+	PERFORMANCE_SCORE	1.6
+	DETAILS:         	[baseline test 3.29988 secs, and comparison 5.37633 sec, and warnIfPerfScore > 0.75, and perfScore=1.62925]
+	                 	Sequence<int> is ***SLOWER***
+
+Test Sequence<string> basics (vector<string> vs Sequence<string>)
+	PERFORMANCE_SCORE	0.094
+	DETAILS:         	[baseline test 11.5729 secs, and comparison 1.08287 sec, and warnIfPerfScore > 0.33, and perfScore=0.0935695]
+	                 	Sequence<string> is FASTER
+
+Test Sequence_DoublyLinkedList<int> basics (vector<int> vs Sequence_DoublyLinkedList<int>)
+	PERFORMANCE_SCORE	12
+	DETAILS:         	[baseline test 3.27711 secs, and comparison 40.495 sec, and warnIfPerfScore > 5.1, and perfScore=12.3569]
+	                 	Sequence_DoublyLinkedList<int> is ***SLOWER***
+
+Test Sequence_Array<int> basics (vector<int> vs Sequence_Array<int>)
+	PERFORMANCE_SCORE	1.6
+	DETAILS:         	[baseline test 3.82272 secs, and comparison 6.22103 sec, and warnIfPerfScore > 0.7, and perfScore=1.62738]
+	                 	Sequence_Array<int> is ***SLOWER***
+
+Test Sequence_stdvector<int> basics (vector<int> vs Sequence_stdvector<int>)
+	PERFORMANCE_SCORE	1.9
+	DETAILS:         	[baseline test 3.26621 secs, and comparison 6.11323 sec, and warnIfPerfScore > 1.1, and perfScore=1.87166]
+	                 	Sequence_stdvector<int> is ***SLOWER***
+
+Test Sequence_DoublyLinkedList<string> basics (vector<string> vs Sequence_DoublyLinkedList<string>)
+	PERFORMANCE_SCORE	0.58
+	DETAILS:         	[baseline test 5.85298 secs, and comparison 3.40819 sec, and warnIfPerfScore > 0.55, and perfScore=0.582301]
+	                 	Sequence_DoublyLinkedList<string> is FASTER
+
+Test Collection<int> basics (vector<int> vs Collection<int>)
+	PERFORMANCE_SCORE	11
+	DETAILS:         	[baseline test 4.98398 secs, and comparison 52.4459 sec, and warnIfPerfScore > 4.9, and perfScore=10.5229]
+	                 	Collection<int> is ***SLOWER***
+
+Test Collection<string> basics (vector<string> vs Collection<string>)
+	PERFORMANCE_SCORE	0.56
+	DETAILS:         	[baseline test 8.19922 secs, and comparison 4.60515 sec, and warnIfPerfScore > 0.85, and perfScore=0.561657]
+	                 	Collection<string> is FASTER
+
+Test Collection_LinkedList<string> basics (vector<string> vs Collection_LinkedList<string>)
+	PERFORMANCE_SCORE	0.42
+	DETAILS:         	[baseline test 8.32511 secs, and comparison 3.49228 sec, and warnIfPerfScore > 0.6, and perfScore=0.419487]
+	                 	Collection_LinkedList<string> is FASTER
+
+Test Collection_stdforward_list<string> basics (vector<string> vs Collection_stdforward_list<string>)
+	PERFORMANCE_SCORE	0.5
+	DETAILS:         	[baseline test 7.18043 secs, and comparison 3.55695 sec, and warnIfPerfScore > 0.6, and perfScore=0.495367]
+	                 	Collection_stdforward_list<string> is FASTER
+
+Test SortedCollection_stdmultiset<string> basics (vector<string> vs SortedCollection_stdmultiset<string>)
+	PERFORMANCE_SCORE	0.58
+	DETAILS:         	[baseline test 7.08256 secs, and comparison 4.08238 sec, and warnIfPerfScore > 1, and perfScore=0.576399]
+	                 	SortedCollection_stdmultiset<string> is FASTER
+
+Test SortedCollection_stdmultiset<string> basics with rnd strings (vector<string> vs SortedCollection_stdmultiset<string>)
+	PERFORMANCE_SCORE	0.69
+	DETAILS:         	[baseline test 12.5434 secs, and comparison 8.64696 sec, and warnIfPerfScore > 1.3, and perfScore=0.689363]
+	                 	SortedCollection_stdmultiset<string> is FASTER
+
+Test std::set<int> vs Set<int> (set<int> vs Set<int>)
+	PERFORMANCE_SCORE	0.29
+	DETAILS:         	[baseline test 16.6489 secs, and comparison 4.82633 sec, and warnIfPerfScore > 0.21, and perfScore=0.28989]
+	                 	Set<int> is FASTER
+
+Test String Characters::Format () (sprintf vs String Characters::Format)
+	PERFORMANCE_SCORE	2.9
+	DETAILS:         	[baseline test 4.79374 secs, and comparison 13.9762 sec, and warnIfPerfScore > 1.8, and perfScore=2.91551]
+	                 	String Characters::Format is ***SLOWER***
+
+Test BLOB versus vector<byte> (vector<byte> vs BLOB)
+	PERFORMANCE_SCORE	0.68
+	DETAILS:         	[baseline test 5.36535 secs, and comparison 3.67262 sec, and warnIfPerfScore > 1, and perfScore=0.684507]
+	                 	BLOB is FASTER
+
+Test BLOB versus vector<byte> ver#2 (vector<byte> vs BLOB)
+	PERFORMANCE_SCORE	0.75
+	DETAILS:         	[baseline test 3.58712 secs, and comparison 2.67313 sec, and warnIfPerfScore > 0.85, and perfScore=0.745201]
+	                 	BLOB is FASTER
+
+Test Test_JSONReadWriteFile (15 seconds vs Test_JSONReadWriteFile)
+	PERFORMANCE_SCORE	0.35
+	DETAILS:         	[baseline test 15 secs, and comparison 5.29042 sec, and warnIfPerfScore > 0.5, and perfScore=0.352695]
+	                 	Test_JSONReadWriteFile is FASTER
+
+Test Test_Optional_ (15 seconds vs Test_Optional_)
+	PERFORMANCE_SCORE	0.08
+	DETAILS:         	[baseline test 15 secs, and comparison 1.19569 sec, and warnIfPerfScore > 0.5, and perfScore=0.0797126]
+	                 	Test_Optional_ is FASTER
+
+stroika-json-native-parser: "./Tests/51/JSONTestData/small-dict.json"
+	DETAILS			0.253474 seconds
+
+stroika-json-native-parser: "./Tests/51/JSONTestData/medium-dict.json"
+	DETAILS			2.56144 seconds
+
+stroika-json-native-parser: "./Tests/51/JSONTestData/large-dict.json"
+	DETAILS			9.51471 seconds
+
+stroika-json-parser: "./Tests/51/JSONTestData/small-dict.json"
+	DETAILS			0.0791956 seconds
+
+stroika-json-parser: "./Tests/51/JSONTestData/medium-dict.json"
+	DETAILS			1.77808 seconds
+
+stroika-json-parser: "./Tests/51/JSONTestData/large-dict.json"
+	DETAILS			5.8862 seconds
+
+boost_json-parser: "./Tests/51/JSONTestData/small-dict.json"
+	DETAILS			0.0232188 seconds
+
+boost_json-parser: "./Tests/51/JSONTestData/medium-dict.json"
+	DETAILS			0.376305 seconds
+
+boost_json-parser: "./Tests/51/JSONTestData/large-dict.json"
+	DETAILS			1.32055 seconds
+
+boost_json-vv-parser: "./Tests/51/JSONTestData/small-dict.json"
+	DETAILS			0.0764539 seconds
+
+boost_json-vv-parser: "./Tests/51/JSONTestData/medium-dict.json"
+	DETAILS			1.46881 seconds
+
+boost_json-vv-parser: "./Tests/51/JSONTestData/large-dict.json"
+	DETAILS			5.9029 seconds
+
+[[[Tests took: 12 minutes, 14 seconds]]]
+