Fix: service LocallyRegisteredAsSidecar property is not persisted #8924
Conversation
ShimmerGlass
force-pushed
the
fix-sidecar-deregister-after-restart
branch
from
bf248fe to
dd08a90
Compare
rboyer
added
the
theme/connect
ShimmerGlass
force-pushed
the
fix-sidecar-deregister-after-restart
branch
from
dd08a90 to
d26e321
Compare
When a service is deregistered, we check whever matching services were registered as sidecar along with it and deregister them as well. To determine if a service is indeed a sidecar we check the structs.ServiceNode.LocallyRegisteredAsSidecar property. However, to avoid interal API leakage, it is excluded from JSON serialization, meaning it is not persisted to disk either. When the agent is restarted, this property lost and sidecars are no longer deregistered along with their parent service. To fix this, we now specifically save this property in the persisted service file.
ShimmerGlass
force-pushed
the
fix-sidecar-deregister-after-restart
branch
from
d26e321 to
1c8369b
Compare
|
@dnephin Added the changelog. Is the wording OK? |
|
@dnephin Did you have time to take a look? Dead sidecars pile up on our side as we restart consul agents pretty often in our PAAS. |
|
🍒✅ Cherry pick of commit 3a55c30 onto |
…er-restart Fix: service LocallyRegisteredAsSidecar property is not persisted
|
Thank you ! |
|
Hi here. I am facing an issue with Consul service mesh upgrade. earlier I am using helm chart 1.6.0 and it was working fine . After upgrading to 1.9.0 I am facing a lot of issues with consul service mesh especially services are not getting de-registered when I deploy a new pod using helm package manager.I am seeing 2 to 3 instances in each service and I am trying to de-register them by logging into the POD, but unable to deregister some of the services.. Due to this my ambassador endpoint API gateway is unable to connect to the services and due to this I am unable to connect to any services..it is causing a big issue.
|
|
@SamaChandra I have encountered the same problem. Have you solved it now？ |
When a service is deregistered, we check whever matching services were
registered as sidecar along with it and deregister them as well.
To determine if a service is indeed a sidecar we check the
structs.ServiceNode.LocallyRegisteredAsSidecar property. However, to
avoid interal API leakage, it is excluded from JSON serialization,
meaning it is not persisted to disk either.
When the agent is restarted, this property lost and sidecars are no
longer deregistered along with their parent service.
To fix this, we now specifically save this property in the persisted
service file.