Code-generate for federated programs #175

erlingrj · 2024-12-13T19:19:10Z

TODOs:

…the build

But invoking CMake correctly to generate two binaries and a shell script in `bin` is not working

github-actions · 2024-12-14T01:52:06Z

Memory usage after merging this PR will be:

Memory Report

action_empty_test_c

	from	to	increase (%)
text	59711	60113	0.67
data	744	752	1.08
bss	10112	11360	12.34
total	70567	72225	2.35

action_microstep_test_c

	from	to	increase (%)
text	60582	60984	0.66
data	752	760	1.06
bss	10112	11424	12.97
total	71446	73168	2.41

action_overwrite_test_c

	from	to	increase (%)
text	60419	60821	0.67
data	744	752	1.08
bss	10112	11424	12.97
total	71275	72997	2.42

action_test_c

	from	to	increase (%)
text	60323	60725	0.67
data	752	760	1.06
bss	10112	11424	12.97
total	71187	72909	2.42

deadline_test_c

	from	to	increase (%)
text	55918	56355	0.78
data	760	768	1.05
bss	10784	10784	0.00
total	67462	67907	0.66

delayed_conn_test_c

	from	to	increase (%)
text	61386	61717	0.54
data	744	752	1.08
bss	10112	10272	1.58
total	72242	72741	0.69

event_payload_pool_test_c

	from	to
text	18331	18331
data	624	624
bss	320	320
total	19275	19275

event_queue_test_c

	from	to	increase (%)
text	27518	27616	0.36
data	736	736	0.00
bss	480	480	0.00
total	28734	28832	0.34

nanopb_test_c

	from	to
text	42886	42886
data	904	904
bss	320	320
total	44110	44110

port_test_c

	from	to	increase (%)
text	61334	61665	0.54
data	744	752	1.08
bss	10112	10272	1.58
total	72190	72689	0.69

reaction_queue_test_c

	from	to	increase (%)
text	27318	27448	0.48
data	736	736	0.00
bss	480	480	0.00
total	28534	28664	0.46

request_shutdown_test_c

	from	to	increase (%)
text	60554	60956	0.66
data	744	752	1.08
bss	10112	11424	12.97
total	71410	73132	2.41

startup_test_c

	from	to	increase (%)
text	55617	56054	0.79
data	752	760	1.06
bss	10784	10784	0.00
total	67153	67598	0.66

tcp_channel_test_c

	from	to	increase (%)
text	93856	95347	1.59
data	1224	1232	0.65
bss	21312	21408	0.45
total	116392	117987	1.37

timer_test_c

	from	to	increase (%)
text	55508	55945	0.79
data	744	752	1.08
bss	10784	10784	0.00
total	67036	67481	0.66

github-actions · 2025-01-08T12:09:04Z

Benchmark results after merging this PR:

Benchmark results

Performance:

PingPongUc:
Best Time: 136.584 msec
Worst Time: 143.416 msec
Median Time: 137.506 msec

PingPongC:
Best Time: 168.343 msec
Worst Time: 177.727 msec
Median Time: 169.557 msec

ReactionLatencyUc:
Best latency: 37979 nsec
Median latency: 60413 nsec
Worst latency: 96334 nsec

ReactionLatencyC:
Best latency: 23997 nsec
Median latency: 60185 nsec
Worst latency: 102418 nsec

Memory usage:

PingPongUc:
text data bss dec hex filename
40428 760 8496 49684 c214 bin/PingPongUc

PingPongC:
text data bss dec hex filename
45826 880 360 47066 b7da bin/PingPongC

ReactionLatencyUc:
text data bss dec hex filename
30265 744 2080 33089 8141 bin/ReactionLatencyUc

ReactionLatencyC:
text data bss dec hex filename
41536 848 360 42744 a6f8 bin/ReactionLatencyC

examples/riot/buildAll.sh

erlingrj · 2025-01-15T15:11:19Z

examples/riot/hello_lf/run/build.sh

@LasseRosenow is this needed?

erlingrj · 2025-01-15T15:12:52Z

include/reactor-uc/environment.h

@@ -9,6 +9,7 @@
 #include "reactor-uc/scheduler.h"

 typedef struct Environment Environment;
+extern Environment *_lf_environment;


I created this global backdoor so that the logging functions could read out the current time and add it to the printouts... we could possibly use this also within the TcpIpChannel and UdpCoapChannel

So you mean we remove the Environment pointer from the constructor and just use the global pointer?

Yes that is possible. It is a little dirty, but, having global access to the environment also makes sense.

erlingrj · 2025-01-15T15:15:30Z

CMakeLists.txt

@@ -78,7 +78,7 @@ if (NOT PLATFORM STREQUAL "ZEPHYR")
 endif()

 # Add compile definitions for platform and network channel specifics.
-target_compile_definitions(reactor-uc PRIVATE "PLATFORM_${PLATFORM}")
+target_compile_definitions(reactor-uc PUBLIC "PLATFORM_${PLATFORM}")


The PLATFORM compile def must be visible in targets that link to reactor-uc also, because they will include the network_channel.h where it is used

erlingrj · 2025-01-15T15:17:55Z

src/environment.c

@@ -49,7 +49,11 @@ interval_t Environment_get_physical_time(Environment *self) {
  return self->platform->get_physical_time(self->platform);
 }
 interval_t Environment_get_elapsed_physical_time(Environment *self) {
-  return self->platform->get_physical_time(self->platform) - self->scheduler->start_time;
+  if (self->scheduler->start_time == NEVER) {


Support reading out elapsed physical time even before we have started

erlingrj · 2025-01-15T15:19:13Z

src/federated.c

@@ -150,7 +155,7 @@ void FederatedInputConnection_ctor(FederatedInputConnection *self, Reactor *pare
  self->delay = delay;
  self->type = type;
  self->last_known_tag = NEVER_TAG;
-  self->safe_to_assume_absent = FOREVER;
+  self->safe_to_assume_absent = 0; // FIXME: This should be set by the user


Note that I change the default here. By setting this to 0 we move from a dataflow federation to a "poor-mans-ptides" where the after-delays on the federated connections should be greater than the actual network delays

erlingrj · 2025-01-15T15:20:06Z

src/federated.c

        break;
      case LF_PAST_TAG:
-        LF_WARN(FED, "Tried scheduling event to a past tag. Dropping\n");
+        LF_ERR(FED, "Safe-to-process violation! Tried scheduling event to a past tag. Handling now instead!");


If we receive an event destined for a past tag, then instead of dropping it, we handle it ASAP. In the future we will add support for a handler for this case, just like the deadlines

erlingrj · 2025-01-15T15:21:09Z

src/platform/posix/posix.c

@@ -29,7 +29,8 @@ static struct timespec convert_ns_to_timespec(instant_t time) {
 }

 lf_ret_t PlatformPosix_initialize(Platform *_self) {
-  signal(SIGINT, handle_ctrlc);
+  signal(SIGINT, handle_signal);


Handle both ctrl-c and SIGTERM and stop the program

erlingrj · 2025-01-15T15:22:03Z

src/platform/posix/tcp_ip_channel.c

                       NetworkChannel_state_to_string(new_state));

+  pthread_mutex_lock(&self->mutex);


Here a mutex is used as a straightforward way of protecting this shared state

Looks good!

github-actions · 2025-01-15T17:25:47Z

Coverage after merging lfc-federated into main will be

71.37%

Coverage Report

File	Stmts	Branches	Funcs	Lines	Uncovered Lines
src
action.c	77.69%	65.63%	100%	81.18%	134–135, 24, 42–45, 48, 50–51, 54–56, 62–63, 70–72, 72, 72–75, 81–82, 93–94
builtin_triggers.c	90.91%	70%	100%	96.77%	14, 18, 40, 43
connection.c	78.52%	51.16%	100%	88.66%	10, 104, 11, 110, 123–124, 136–137, 14, 14, 143, 145, 16–17, 21–22, 22, 22–23, 25, 27–28, 33, 48, 48, 48–49, 55, 60–62, 97
environment.c	82.18%	60%	92.31%	86.76%	12–13, 18, 20–21, 31, 35–36, 42–43, 59–60, 64–65, 97–99
event.c	95.35%	92.86%	100%	96.15%	14–15
federated.c	5.44%	2.91%	7.69%	6.51%	10, 100, 100, 100–101, 104, 107–108, 108, 108–109, 111–112, 114, 118–119, 12, 121–123, 126, 128–133, 135–137, 14, 14, 14, 140–142, 142, 142–143, 143, 143–145, 147, 15, 150–151, 153–157, 159, 16, 160–164, 166, 166, 166–169, 17, 171, 171, 171–173, 173, 173–174, 178–179, 179, 179, 18, 18, 18, 182–183, 187–189, 191, 191, 191, 193–197, 200, 200, 200–203, 206–207, 207, 207–208, 21, 210–211, 214–215, 22, 220–221, 221, 221–222, 224, 226, 226, 226–229, 229, 229, 229, 229, 23, 23, 23, 230–238, 238, 238–239, 24, 241, 243–249, 25, 25, 25, 253, 256, 256, 256–258, 26, 262, 265–266, 266, 266, 266–269, 27, 270–274, 276, 28, 28, 28, 282–284, 29, 29, 29, 29, 29, 296–297, 30, 300–303, 305, 305, 305–306, 310–311, 311, 311, 313, 315–316, 316, 316–317, 317, 317–318, 318, 318–319, 319, 319, 32, 320, 320, 320–321, 321, 321, 323, 323, 323–324, 324, 324–325, 325, 325–326, 326, 326, 328, 35, 35, 35, 35, 35–36, 40–41, 45–46, 48–51, 53, 53, 53–54, 54, 54, 56, 56, 56–58, 58, 58–60, 64–65, 69–70, 72–75, 77, 79, 79, 79–80, 80, 80–81, 81, 81–82, 82, 82, 85–86, 88–89, 9, 90–91, 93, 93, 93–96, 98
logging.c	88.52%	83.33%	100%	89.36%	25, 38–40, 47, 60–61
network_channel.c	69.23%	62.50%	100%	70.59%	40, 40, 40, 45–48, 57
port.c	78.08%	45.83%	100%	93.33%	10, 10, 10, 16, 20, 25, 25–27, 27, 27–28, 39, 39, 39–40
queues.c	89.94%	80.36%	100%	94.06%	108, 113, 119, 21–23, 47–48, 60–61, 84–88, 91–92
reaction.c	71.19%	54.55%	100%	79.71%	15, 17, 21, 28–31, 31, 31–32, 42, 45, 47, 52–53, 53, 53–55, 55, 55–56, 73, 89–91, 91, 91–94, 94, 94–95
reactor.c	69.33%	51.52%	100%	82.28%	10, 101–102, 14–19, 22, 28, 30, 32–37, 37, 37–38, 38, 38, 43, 55, 58–59, 59, 59–60, 60, 60–61, 63, 77–78, 81–82, 82, 82–83, 83, 83–84, 86, 91
serialization.c	50%	50%	50%	50%	16–17, 26–27, 33–35, 38–40
tag.c	40.19%	31.48%	60%	47.92%	14, 14–15, 17, 17–18, 23–24, 24, 24, 24, 24–25, 27, 27, 27, 27, 27–28, 30, 30, 30–31, 33–34, 34, 34–35, 37, 37, 37, 37, 37–38, 40, 40, 40, 40, 40–41, 43, 53–54, 63, 63–64, 83–85, 85, 85, 85, 85, 85, 85, 85, 85, 85, 85–87, 89
timer.c	95%	66.67%	100%	100%	14, 25
trigger.c	100%	100%	100%	100%
util.c	41.67%	33.33%	33.33%	46.67%	12–13, 13, 13–16, 18–20, 4–5
src/platform/posix
posix.c	53.15%	30%	66.67%	56.58%	100–101, 101, 101–103, 107, 16, 18, 20–21, 35–37, 39–41, 49–50, 55–60, 60, 60–63, 63, 63–65, 68, 74–75, 79, 82, 93–95, 95, 95–97, 99
tcp_ip_channel.c	66.06%	51.50%	100%	75.59%	106–107, 109, 111, 115–116, 124, 127–128, 128, 128–129, 134–135, 135, 135–136, 142–143, 143, 143–145, 159, 165, 165, 165, 165, 165–166, 166, 166–167, 167, 167–168, 170, 172, 172, 172–173, 182, 189–190, 194, 198, 198, 198–200, 200, 200–201, 203, 203, 203–204, 204, 204, 206, 208–209, 213, 213, 213–214, 234, 247–248, 248, 248–249, 255, 260–261, 261, 261–262, 262, 262–263, 265–266, 266, 266–267, 267, 267, 269–272, 282, 282–283, 283, 283–284, 308–309, 309, 309–311, 311, 311–312, 312, 312–313, 322,

erlingrj and others added 11 commits December 8, 2024 09:05

Have LFC generate a main function that can optionally be included in …

0584253

…the build

CI

6b51697

CI

55b308a

CI

a8973e8

Setup LFC in RIOT container

cfda3bd

Switch to riot master

2c38b98

Fix

68e8d11

Install LFC deps in zephyr CI also

60ceeea

WIP

2704dbd

WIP

5ba1392

WIP: SimpleFederated.lf now compiles.

a8bc451

But invoking CMake correctly to generate two binaries and a shell script in `bin` is not working

erlingrj marked this pull request as draft December 13, 2024 19:19

erlingrj added 3 commits December 13, 2024 15:28

Generate a launch script for federated native

ead4a67

Merge remote-tracking branch 'origin/main' into lfc-federated

ddbed66

Make federated launch script executable

aec7438

erlingrj added 6 commits December 20, 2024 10:00

WIP

d0b4e22

Rework the handling of connections

8efd065

Various fixes to get all standalone tests to pass again

c049c73

Formatting

991264a

More WIP

43874dd

Formatting

3689883

erlingrj added 7 commits January 9, 2025 15:12

More WIP

9eb722d

Add @interface attr

d5e95fe

More WIP

9b63d84

More WIP

2f9c0f4

More WIP

03a428c

Merge remote-tracking branch 'origin/main' into lfc-federated

c7dde4f

All tests are passing

f910e5e

erlingrj added 4 commits January 13, 2025 21:37

Refactor

841549f

Formatting

d3173cf

Refactorings

dd61844

More docs

42fc1ad

erlingrj changed the title ~~[DRAFT] Code-generate for federated programs~~ Code-generate for federated programs Jan 15, 2025

CI

9fee19b

erlingrj marked this pull request as ready for review January 15, 2025 14:50

erlingrj commented Jan 15, 2025

View reviewed changes

examples/riot/buildAll.sh Outdated Show resolved Hide resolved

erlingrj commented Jan 15, 2025

View reviewed changes

examples/riot/hello_lf/run/build.sh Outdated

Copy link

Collaborator Author

erlingrj Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LasseRosenow is this needed?

erlingrj commented Jan 15, 2025

View reviewed changes

Fixes

97bb16a

erlingrj commented Jan 15, 2025

View reviewed changes

erlingrj added 8 commits January 15, 2025 16:27

TcpIp fixes

d56ce51

Revert more tcp stuff

e2176a2

Avoid flooding log when a federate closes a socket

7b49934

Remove merge mistake

6b5e91d

Formatting

9e79137

Fix posix federated

8d51127

More minor fixes

b906004

Format

ef9f980

erlingrj added 2 commits January 15, 2025 19:05

Minimum event queue of 2

05d7d87

Remove some dead code

42166cd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Code-generate for federated programs #175

Code-generate for federated programs #175

erlingrj commented Dec 13, 2024 •

edited

Loading

github-actions bot commented Dec 14, 2024 •

edited

Loading

action_empty_test_c

action_microstep_test_c

action_overwrite_test_c

action_test_c

deadline_test_c

delayed_conn_test_c

event_payload_pool_test_c

event_queue_test_c

nanopb_test_c

port_test_c

reaction_queue_test_c

request_shutdown_test_c

startup_test_c

tcp_channel_test_c

timer_test_c

github-actions bot commented Jan 8, 2025 •

edited

Loading

Performance:

Memory usage:

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

LasseRosenow Jan 15, 2025

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

erlingrj Jan 15, 2025

LasseRosenow Jan 15, 2025

github-actions bot commented Jan 15, 2025

		NetworkChannel_state_to_string(new_state));

		pthread_mutex_lock(&self->mutex);

Code-generate for federated programs #175

Are you sure you want to change the base?

Code-generate for federated programs #175

Conversation

erlingrj commented Dec 13, 2024 • edited Loading

github-actions bot commented Dec 14, 2024 • edited Loading

action_empty_test_c

action_microstep_test_c

action_overwrite_test_c

action_test_c

deadline_test_c

delayed_conn_test_c

event_payload_pool_test_c

event_queue_test_c

nanopb_test_c

port_test_c

reaction_queue_test_c

request_shutdown_test_c

startup_test_c

tcp_channel_test_c

timer_test_c

github-actions bot commented Jan 8, 2025 • edited Loading

Performance:

Memory usage:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jan 15, 2025

erlingrj commented Dec 13, 2024 •

edited

Loading

github-actions bot commented Dec 14, 2024 •

edited

Loading

github-actions bot commented Jan 8, 2025 •

edited

Loading