Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use target dir for integration tests TP #1342

Closed
wants to merge 1 commit into from
Closed

use target dir for integration tests TP #1342

wants to merge 1 commit into from

Conversation

plebhash
Copy link
Collaborator

@plebhash plebhash commented Jan 9, 2025

close #1278

unfortunately #1293 didn't fix it

this PR changes the directory where TP is downloaded into, replacing /tmp (which was giving permission errors in an undeterministic way) to the target directory crated by cargo (which shouldn't ever have permission issues)

@plebhash plebhash changed the title use target dir for integration tests TP.. use target dir for integration tests TP Jan 9, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintegration_tests_tp_target_dir
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,521.30
(-0.53%)		6,932.83
(94.06%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,212.30
(-2.56%)		7,867.13
(91.68%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		7,904.50
(-2.29%)		9,387.58
(84.20%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		893.11
(+2.80%)		945.48
(94.46%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		690.53
(+2.04%)		725.47
(95.18%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		246.95
(-1.19%)		273.61
(90.25%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		158.13
(+0.39%)		165.20
(95.72%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,319.20
(-0.42%)		6,771.74
(93.32%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		277.82
(-2.10%)		336.45
(82.57%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		728.07
(+0.10%)		778.45
(93.53%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		589.90
(-0.13%)		627.66
(93.98%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		204.08
(-1.10%)		226.09
(90.26%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintegration_tests_tp_target_dir
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
1e3 x estimated cycles
(Result Δ%)		Upper Boundary
1e3 x estimated cycles
(Limit %)		InstructionsBenchmark Result
1e3 x instructions
(Result Δ%)		Upper Boundary
1e3 x instructions
(Limit %)		L1 AccessesBenchmark Result
1e3 x accesses
(Result Δ%)		Upper Boundary
1e3 x accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8.42
(-0.40%)		8.67
(97.11%)		📈 view plot
🚷 view threshold		3.69
(-0.93%)		3.86
(95.53%)		📈 view plot
🚷 view threshold		5.16
(-1.07%)		5.45
(94.66%)		📈 view plot
🚷 view threshold		9.00
(+5.59%)		16.42
(54.80%)		📈 view plot
🚷 view threshold		92.00
(+0.63%)		96.82
(95.02%)
get_submit📈 view plot
🚷 view threshold		95.22
(-0.14%)		95.61
(99.59%)		📈 view plot
🚷 view threshold		59.35
(-0.09%)		59.71
(99.39%)		📈 view plot
🚷 view threshold		85.22
(-0.10%)		85.82
(99.30%)		📈 view plot
🚷 view threshold		47.00
(+4.83%)		60.38
(77.84%)		📈 view plot
🚷 view threshold		279.00
(-0.62%)		291.87
(95.59%)
get_subscribe📈 view plot
🚷 view threshold		7.90
(-1.14%)		8.23
(96.02%)		📈 view plot
🚷 view threshold		2.77
(-1.65%)		2.94
(93.97%)		📈 view plot
🚷 view threshold		3.85
(-1.90%)		4.14
(92.87%)		📈 view plot
🚷 view threshold		13.00
(+3.55%)		20.74
(62.67%)		📈 view plot
🚷 view threshold		114.00
(-0.46%)		117.92
(96.67%)
serialize_authorize📈 view plot
🚷 view threshold		12.14
(-0.94%)		12.50
(97.10%)		📈 view plot
🚷 view threshold		5.27
(-0.59%)		5.43
(97.05%)		📈 view plot
🚷 view threshold		7.33
(-0.68%)		7.60
(96.38%)		📈 view plot
🚷 view threshold		11.00
(+6.99%)		18.90
(58.19%)		📈 view plot
🚷 view threshold		136.00
(-1.42%)		143.32
(94.89%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24.61
(-0.44%)		25.19
(97.70%)		📈 view plot
🚷 view threshold		9.84
(-0.19%)		10.01
(98.28%)		📈 view plot
🚷 view threshold		13.88
(-0.22%)		14.18
(97.89%)		📈 view plot
🚷 view threshold		39.00
(+8.40%)		45.97
(84.83%)		📈 view plot
🚷 view threshold		301.00
(-0.86%)		313.67
(95.96%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		30.07
(-0.82%)		30.73
(97.87%)		📈 view plot
🚷 view threshold		12.02
(-0.28%)		12.19
(98.60%)		📈 view plot
🚷 view threshold		17.00
(-0.32%)		17.30
(98.30%)		📈 view plot
🚷 view threshold		59.00
(+5.59%)		67.50
(87.41%)		📈 view plot
🚷 view threshold		365.00
(-1.62%)		379.52
(96.17%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126.16
(-0.23%)		126.79
(99.50%)		📈 view plot
🚷 view threshold		73.20
(-0.06%)		73.53
(99.55%)		📈 view plot
🚷 view threshold		104.92
(-0.08%)		105.51
(99.44%)		📈 view plot
🚷 view threshold		111.00
(+3.95%)		125.93
(88.15%)		📈 view plot
🚷 view threshold		591.00
(-1.13%)		610.70
(96.77%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27.66
(-0.88%)		28.39
(97.46%)		📈 view plot
🚷 view threshold		9.58
(-0.47%)		9.76
(98.20%)		📈 view plot
🚷 view threshold		13.54
(-0.54%)		13.84
(97.84%)		📈 view plot
🚷 view threshold		67.00
(+3.73%)		78.01
(85.88%)		📈 view plot
🚷 view threshold		394.00
(-1.31%)		409.82
(96.14%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115.08
(-0.14%)		115.72
(99.45%)		📈 view plot
🚷 view threshold		68.06
(+0.01%)		68.42
(99.47%)		📈 view plot
🚷 view threshold		97.65
(+0.01%)		98.29
(99.35%)		📈 view plot
🚷 view threshold		69.00
(+5.09%)		87.79
(78.60%)		📈 view plot
🚷 view threshold		488.00
(-1.10%)		505.39
(96.56%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		23.15
(-0.76%)		23.82
(97.18%)		📈 view plot
🚷 view threshold		8.14
(-0.51%)		8.32
(97.92%)		📈 view plot
🚷 view threshold		11.45
(-0.60%)		11.75
(97.48%)		📈 view plot
🚷 view threshold		43.00
(+10.06%)		51.20
(83.98%)		📈 view plot
🚷 view threshold		328.00
(-1.10%)		342.48
(95.77%)
serialize_submit📈 view plot
🚷 view threshold		99.61
(-0.17%)		100.09
(99.52%)		📈 view plot
🚷 view threshold		61.41
(-0.07%)		61.73
(99.47%)		📈 view plot
🚷 view threshold		88.08
(-0.08%)		88.65
(99.36%)		📈 view plot
🚷 view threshold		52.00
(+7.77%)		67.11
(77.49%)		📈 view plot
🚷 view threshold		322.00
(-1.05%)		337.63
(95.37%)
serialize_subscribe📈 view plot
🚷 view threshold		11.35
(-0.45%)		11.60
(97.84%)		📈 view plot
🚷 view threshold		4.12
(-1.03%)		4.28
(96.15%)		📈 view plot
🚷 view threshold		5.71
(-1.25%)		6.00
(95.17%)		📈 view plot
🚷 view threshold		15.00
(+7.20%)		24.31
(61.70%)		📈 view plot
🚷 view threshold		159.00
(+0.28%)		164.64
(96.58%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintegration_tests_tp_target_dir
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.08
(-2.48%)		59.59
(73.98%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		74.75
(-3.26%)		106.71
(70.05%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.65
(-0.10%)		14.72
(99.51%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		260.54
(-1.28%)		288.10
(90.43%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		609.12
(-1.34%)		679.77
(89.61%)
client_sv2_open_channel📈 view plot
🚷 view threshold		164.77
(-0.66%)		179.05
(92.03%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		277.29
(-2.97%)		314.95
(88.04%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		377.22
(-1.40%)		402.53
(93.71%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		162.47
(+1.47%)		171.41
(94.79%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		461.63
(-1.62%)		553.72
(83.37%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		984.61
(-2.35%)		1,218.29
(80.82%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintegration_tests_tp_target_dir
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
1e3 x estimated cycles
(Result Δ%)		Upper Boundary
1e3 x estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2.16
(+2.26%)		2.24
(96.63%)		📈 view plot
🚷 view threshold		473.00
(-0.13%)		490.59
(96.41%)		📈 view plot
🚷 view threshold		733.00
(-0.40%)		758.98
(96.58%)		📈 view plot
🚷 view threshold		6.00
(+17.97%)		11.60
(51.71%)		📈 view plot
🚷 view threshold		40.00
(+3.41%)		41.76
(95.79%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8.35
(+1.51%)		8.40
(99.39%)		📈 view plot
🚷 view threshold		2,137.00📈 view plot
🚷 view threshold		3,152.00
(-0.21%)		3,167.52
(99.51%)		📈 view plot
🚷 view threshold		39.00
(+9.55%)		41.60
(93.75%)		📈 view plot
🚷 view threshold		143.00
(+2.34%)		144.65
(98.86%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6.42
(+1.87%)		6.45
(99.50%)		📈 view plot
🚷 view threshold		1,750.00
(-0.03%)		1,767.59
(99.00%)		📈 view plot
🚷 view threshold		2,546.00
(-0.23%)		2,575.47
(98.86%)		📈 view plot
🚷 view threshold		19.00
(+10.75%)		24.05
(79.00%)		📈 view plot
🚷 view threshold		108.00
(+3.12%)		108.90
(99.17%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14.86
(+0.89%)		14.94
(99.45%)		📈 view plot
🚷 view threshold		4,694.00
(-0.01%)		4,711.59
(99.63%)		📈 view plot
🚷 view threshold		6,743.00
(-0.19%)		6,786.77
(99.36%)		📈 view plot
🚷 view threshold		55.00
(+20.01%)		63.39
(86.77%)		📈 view plot
🚷 view threshold		224.00
(+1.26%)		226.21
(99.02%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		27.86
(+0.93%)		28.07
(99.28%)		📈 view plot
🚷 view threshold		10,645.00
(+0.35%)		10,699.43
(99.49%)		📈 view plot
🚷 view threshold		15,499.00
(+0.37%)		15,595.25
(99.38%)		📈 view plot
🚷 view threshold		93.00
(+10.92%)		99.35
(93.61%)		📈 view plot
🚷 view threshold		340.00
(+1.32%)		343.30
(99.04%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4.49
(+1.93%)		4.59
(97.91%)		📈 view plot
🚷 view threshold		1,461.00
(-0.04%)		1,478.59
(98.81%)		📈 view plot
🚷 view threshold		2,154.00
(-0.30%)		2,184.77
(98.59%)		📈 view plot
🚷 view threshold		12.00
(+46.01%)		14.07
(85.30%)		📈 view plot
🚷 view threshold		65.00
(+3.29%)		67.83
(95.83%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14.08
(+0.41%)		14.20
(99.12%)		📈 view plot
🚷 view threshold		5,064.00
(-0.01%)		5,081.59
(99.65%)		📈 view plot
🚷 view threshold		7,318.00
(-0.11%)		7,352.79
(99.53%)		📈 view plot
🚷 view threshold		43.00
(+17.13%)		48.53
(88.61%)		📈 view plot
🚷 view threshold		187.00
(+0.52%)		190.91
(97.95%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22.94
(+1.00%)		23.08
(99.39%)		📈 view plot
🚷 view threshold		8,040.00
(+0.09%)		8,057.64
(99.78%)		📈 view plot
🚷 view threshold		11,683.00
(-0.02%)		11,713.47
(99.74%)		📈 view plot
🚷 view threshold		88.00
(+15.94%)		90.27
(97.48%)		📈 view plot
🚷 view threshold		309.00
(+1.58%)		312.95
(98.74%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4.75
(+1.20%)		4.79
(99.08%)		📈 view plot
🚷 view threshold		1,502.00
(-0.04%)		1,519.59
(98.84%)		📈 view plot
🚷 view threshold		2,274.00
(-0.21%)		2,300.99
(98.83%)		📈 view plot
🚷 view threshold		12.00
(+27.15%)		15.31
(78.37%)		📈 view plot
🚷 view threshold		69.00
(+2.05%)		70.18
(98.32%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16.26
(+0.61%)		16.33
(99.55%)		📈 view plot
🚷 view threshold		5,963.00
(-0.01%)		5,980.59
(99.71%)		📈 view plot
🚷 view threshold		8,651.00
(-0.15%)		8,692.71
(99.52%)		📈 view plot
🚷 view threshold		51.00
(+25.13%)		56.52
(90.23%)		📈 view plot
🚷 view threshold		210.00
(+0.83%)		212.20
(98.96%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35.76
(+0.42%)		35.96
(99.44%)		📈 view plot
🚷 view threshold		14,888.00
(+0.14%)		14,919.71
(99.79%)		📈 view plot
🚷 view threshold		21,866.00
(+0.10%)		21,919.07
(99.76%)		📈 view plot
🚷 view threshold		111.00
(+17.26%)		120.22
(92.33%)		📈 view plot
🚷 view threshold		381.00
(+0.36%)		385.75
(98.77%)
🐰 View full continuous benchmarking report in Bencher

@plebhash
use target dir for integration tests TP..
0abd6f2 
using /tmp is leading to undeterministic behavior on Github CI
@plebhash plebhash force-pushed the integration_tests_tp_target_dir branch from 24a36f1 to 0abd6f2 Compare January 9, 2025 16:50
@jbesraa
Copy link
Contributor

jbesraa commented Jan 9, 2025

Ha, it is happening all over the place suddenly.

@codecov Codecov
Copy link

codecov bot commented Jan 9, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.14%. Comparing base (a053dc7) to head (0abd6f2).
Report is 31 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1342   +/-   ##
=======================================
  Coverage   19.14%   19.14%           
=======================================
  Files         166      166           
  Lines       10987    10987           
=======================================
  Hits         2104     2104           
  Misses       8883     8883
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.65% <ø> (ø)
binary_sv2-coverage 5.48% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (ø)
buffer_sv2-coverage 25.02% <ø> (ø)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage 0.00% <ø> (ø)
framing_sv2-coverage 0.29% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.51% <ø> (+0.01%) ⬆️
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.35% <ø> (ø)
pool_sv2-coverage 2.05% <ø> (ø)
protocols 24.74% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 8.11% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils 25.13% <ø> (ø)
v1-coverage 2.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jbesraa
jbesraa suggested changes Jan 16, 2025
View reviewed changes
Copy link
Contributor

@jbesraa jbesraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nack. writing to the same folder cargo writes to can lead to unexpected behavior. it would also make the local size of the project much bigger.
maybe create a specific folder under tests-integration and clean it after the TemplateProvider is dropped.

@plebhash plebhash mentioned this pull request Jan 18, 2025
Merged
@plebhash
Copy link
Collaborator Author

closing in favor of #1359

@plebhash plebhash closed this Jan 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbesraa jbesraa jbesraa requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Test TemplateProvider OS E13
2 participants
@plebhash @jbesraa