-
Notifications
You must be signed in to change notification settings - Fork 532
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Aspire 9 fails to stop projects #6704
Comments
Adding sample exceptions:
and
|
A coworker of mine has been getting similar issues. In that case they were running an ARM MacOS (not sure of the OS version right now) device and starting the app host using Upon stopping and starting the app host, all the C# projects were failing to start since the ports were already in use. |
### Summary & Motivation Aspire is now configured to keep SQL Server alive between debug sessions using the new Aspire 9 `.WithLifetime(ContainerLifetime.Persistent)`, significantly improving secondary startup. Unfortunately, Aspire does not seem to release ports immediately upon shutdown, requiring 10 seconds to a minute before the AppHost can restart. To mitigate this issue temporarily, logic has been added to kill any Developer Control Pane processes (`dcpctrl`) from previous sessions when starting the AppHost, preventing conflicts and strange errors. For more details, see this issue: dotnet/aspire#6704. The SPA generation and `index.html` caching strategy have been updated to align with the new Aspire startup behavior during debugging. The AppGateway no longer launches automatically when starting Aspire AppHost, as it needs a moment to be ready. Additionally, a bug has been resolved where the frontend incorrectly used the old `platformplatform.pfx` certificate instead of `localhost.pfx`. Lastly, a fix for a duplicated `dotnet dotnet` issue in the README has been applied. ### Checklist - [x] I have added a Label to the pull-request - [x] I have added tests, and done manual regression tests - [x] I have updated the documentation, if necessary
I have the same issue. It seems like the Developer Control Pane takes up to 1 minute to shut down. I made an ugly hack to find and kill any hanging control pane as the first step when starting the AppHost. See this commit: platformplatform/PlatformPlatform@1772f94 |
If you are impacted by this, it MAY help to edit your AppHost/Properties/launchSettings.json file between debugger runs to change the Dashboard port number between launches (I bump the port number up or down by 1 each time). This gives the Dashboard extra time to reset (others have reported the Dashboard is the worst/longest duration stuck port offender here). If you have multiple other projects also getting stuck, this unfortunately won't unblock the others. |
That’s what I see too. If I restart before the DCP process is ready, it sometimes takes more than a 1 minute to release the ports... but if you wait 10–15 seconds before starting, it almost always releases. In Aspire 8.2, this used to be 3-5 seconds, and in Aspire 8.0 this was not a problem at all. The code below works for me (only tested on macOS). The only problem is that the browser starts immediately, and the dashboard is not properly shown until I manually hit refresh. using System.Diagnostics;
using AppHost;
using Projects;
// Detect if Aspire ports from the previous run are released. See https://github.com/dotnet/aspire/issues/6704
EnsureDeveloperControlPaneIsNotRunning();
var builder = DistributedApplication.CreateBuilder(args);
// Aspire configuraiton here...
await builder.Build().RunAsync();
void EnsureDeveloperControlPaneIsNotRunning()
{
const string processName = "dcpctrl"; // The Aspire Developer Control Pane process name
var process = Process.GetProcesses()
.SingleOrDefault(p => p.ProcessName.Contains(processName, StringComparison.OrdinalIgnoreCase));
if (process == null) return;
Console.WriteLine($"Shutting down developer control pane from previous run. Process: {process.ProcessName} (ID: {process.Id})");
Thread.Sleep(TimeSpan.FromSeconds(5)); // Allow Docker containers to shut down to avoid orphaned containers
try
{
process.Kill();
Console.WriteLine($"Process {process.Id} killed successfully.");
}
catch (Exception ex)
{
Console.WriteLine($"Failed to kill process {process.Id}: {ex.Message}");
}
} |
@tjementum's work around has been working great for me |
Also impacted by this. Quickly stopping and restarting the application might get you in the reconciliation hell. stop, wait and start seems to solve it. Happy to provide logs if needed. |
Using the workaround of @tjementum as well by opening the task manager and filtering the processes on dcpctrl and ending the task before starting the application again. |
@dbreshears putting this in 9.1 |
@yoDon I would appreciate a report if the latest bits in the Aspire @Kralizek yes, please, if you could share the logs, that would be very helpful @tjementum @thomasvdb would you be wiling to share some detailed logs with us? |
Please find my log below. It's very easy to reproduce:
Please note that all ports are hardcoded. It is not a good developer experience when Aspire changes all ports when AppHost is restarted. If the ports were changed each time, this issue would be resolved. Here is a recording.
|
My system behaves exactly the same as shown by @tjementum |
Are you using Rider as well @Kralizek ? Can you reproduce this using dotnet run as well? |
Also using Rider. Never occurred in console, mostly because i use dotnet watch in console so i don't need stop/start sp quickly |
We're trying to narrow down if this is a Rider issue or an aspire orchestration issue. So far it looks like the common denominator so far is rider (or some combination). Can you see if you can reproduce the issue on the command line with dotnet run? Watch has other problems. |
I confirm I wasn't able to reproduce the issue with dotnet run.
|
Here using Rider as well with this plugin: https://github.com/JetBrains/aspire-plugin |
I was able to reproduce the issue with Visual Studio after a few tries. Investigating... |
But I would not consider this fixed if the only change is randomizing the ports. In some cases I want hardcoded ports to, for example, make it easier to run tests against the locally running system. |
#6931 should have fixed this, but I would appreciate a report if people still see issues with projects not stopping as they should |
@karolz-ms when will this be released? we are very unpatently waiting for the fix :'D |
@andrem0 I hear you. You can try the fix today with the Aspire daily builds https://github.com/dotnet/aspire/blob/main/docs/using-latest-daily.md As for the "official" release, I think the plan of record is to have one sometime in the second half of January (plans might change). |
@karolz-ms I tried daily builds but its still not fixed for me... on m4 mac - rider.. this issue is blocking my company from switching to aspire... suggested solution above also doesnt work all time |
@feritzcan2 does it reproduce when the app is run from the command line ( |
@karolz-ms I cant share from our project unfortunatelly but we have a lot of services and we created a layer in top of aspire. |
@feritzcan2 understood, no worries. It would be really interesting to understand if the issue happens when the app host is started from command line (on the Mac), so we can differentiate between this being Aspire/DCP issue vs Aspire Rider plugin issue. What are the errors you see in the app host output? |
Those are part of errors info: Aspire.Hosting.Dcp.dcpctrl.ExecutableReconciler[0] |
yes I got the same from commandd line dotnet run @karolz-ms |
@feritzcan2 thank you When you start the application, please take a note the path to the If that checks out, would you be willing to send me detailed execution logs? You can reach me at |
@karolz-ms yes it seems like its not using latest version. But we have nuget config and I set dotnet9 source there. What else should I do? |
Thanks @feritzcan2 To verify, I followed the instructions in https://github.com/dotnet/aspire/blob/main/docs/using-latest-daily.md. I do recommend to create a local After the application is built and started search for > ps aux | grep dcp
...
karolz 37808 0.0 0.2 35441568 52788 s005 S 8:59AM 0:01.62
/Users/karolz/.nuget/packages/aspire.hosting.orchestration.osx-x64/9.1.0-preview.1.24619.1/tools/dcp
start-apiserver --monitor 37740 (other parameters)
Verify the version > /Users/karolz/.nuget/packages/aspire.hosting.orchestration.osx-x64/9.1.0-preview.1.24619.1/tools/dcp version
{"version":"0.9.2","commitHash":"e1586b9dd7b6b4bb15cf0b81000ba8ec55854887","buildTimestamp":"2024-12-11T17:11:30Z"}
Here is my local nuget.config, note the <?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<!--To inherit the global NuGet package sources remove the <clear/> line below -->
<clear />
<add key="nuget" value="https://api.nuget.org/v3/index.json" />
<add key="dotnet9" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9/nuget/v3/index.json" />
</packageSources>
</configuration> ... and the AppHost project, note SDK and Aspire.Hosting.AppHost references <Project Sdk="Microsoft.NET.Sdk">
<Sdk Name="Aspire.AppHost.Sdk" Version="9.1.0-preview.1.24619.1" />
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsAspireHost>true</IsAspireHost>
<UserSecretsId>93445db8-0185-405d-8d0e-84e3c7e02bd1</UserSecretsId>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\starterapp.ApiService\starterapp.ApiService.csproj" />
<ProjectReference Include="..\starterapp.Web\starterapp.Web.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Aspire.Hosting.AppHost" Version="9.1.0-preview.1.24619.1" />
</ItemGroup>
</Project> Hope this helps! LMK if after switching ot DCP |
@karolz-ms unfortunatelly still not working :( info: Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler[0] |
@feritzcan2 is this the only unusual message that you see? Note that this is an "info" message, not an error--the app should still work, although connecting to Regarding port that the message mentions (52258):
|
@karolz-ms Hey karol, unfortunatelly there is also error that i didnt share... It happens almost everytime I stop and start immediately, and cannot send requests to the ports that I m using. I'm sharing only part of log but that fail logs exist for all services fail: Aspire.Hosting.Dcp.dcpctrl.ServiceReconciler[0] |
Running that command for DCP process running also validates I'm running 9.1.0 ps xuwww -p 35241 USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND |
@feritzcan2 thank you. Would you be willing to send me detailed logs from the failing run? Email me please for instructions if yes--thanks in advance! |
@karolz-ms yeah I also reallywant to see this is fixed , sending you an email |
Status update: I got the logs from @feritzcan2 (many thanks!) that revealed another case where the Executable shutdown is not done as quickly as it can and should, which causes subsequent run to time out trying to acquire the TCP port. Working on a fix... |
I somehow feel this issue is really important. We are using asp.net mvc with aspire and the developer satisfaction right now is not good. On the one side Thanks for taking a look. |
@AndiRudi agreed. @feritzcan2 helped me test the fix (many thanks!) which should resolve the issue for good; that fix should hit Aspire main branch sometime later this week, so at least you will be able to use Aspire daily builds with the fix until a new Aspire release is made available. I will everybody know via this issue when that happens.
|
What are the symptoms of the I see some strange errors at least once per session asking me to reload the application but I see the application is still running and being updated as usual. |
See dotnet/sdk#36971 |
Should be fixed (Aspire |
Is there an existing issue for this?
Describe the bug
Aspire 9 seems to have lots of problems stopping projects.
In Aspire 8, I routinely started and stopped solutions without issue.
In Aspire 9 on OSX with Podman Desktop and the Rider Aspire plugin, about half the time there are stuck ports preventing Aspire projects from running correctly. I don't know about other operating systems.
All of this is somewhat anecdotal, as I don't have a repro that happens every time, I just know I commonly have the problem. That said, I think closing open browser tabs from the previous session may reduce the incidence of this issue (that was not required with Aspire 8).
Expected Behavior
Starting, stopping, and restarting Aspire projects should "just work", including regardless of whether there are browser tabs left open from a previous run.
Steps To Reproduce
No response
Exceptions (if any)
No response
.NET Version info
IMPORTANT NOTE: I'm running via JetBrains Rider 2024.3 Build #RD-243.21565.191 on OSX, which I believe contains its own dotnet that doesn't show up using dotnet --info.
That said, here is my dotnet --info:
.NET SDK:
Version: 8.0.201
Commit: 4c2d78f037
Workload version: 8.0.200-manifests.2772ffde
Runtime Environment:
OS Name: Mac OS X
OS Version: 14.6
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/8.0.201/
.NET workloads installed:
[aspire]
Installation Source: SDK 8.0.200
Manifest Version: 8.2.1/8.0.100
Manifest Path: /usr/local/share/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.aspire/8.2.1/WorkloadManifest.json
Install Type: FileBased
Host:
Version: 8.0.4
Architecture: arm64
Commit: 2d7eea2529
.NET SDKs installed:
6.0.401 [/usr/local/share/dotnet/sdk]
6.0.403 [/usr/local/share/dotnet/sdk]
6.0.404 [/usr/local/share/dotnet/sdk]
6.0.405 [/usr/local/share/dotnet/sdk]
6.0.407 [/usr/local/share/dotnet/sdk]
6.0.408 [/usr/local/share/dotnet/sdk]
6.0.410 [/usr/local/share/dotnet/sdk]
6.0.413 [/usr/local/share/dotnet/sdk]
6.0.415 [/usr/local/share/dotnet/sdk]
7.0.100 [/usr/local/share/dotnet/sdk]
7.0.101 [/usr/local/share/dotnet/sdk]
7.0.102 [/usr/local/share/dotnet/sdk]
7.0.202 [/usr/local/share/dotnet/sdk]
7.0.203 [/usr/local/share/dotnet/sdk]
7.0.304 [/usr/local/share/dotnet/sdk]
7.0.307 [/usr/local/share/dotnet/sdk]
7.0.410 [/usr/local/share/dotnet/sdk]
8.0.100 [/usr/local/share/dotnet/sdk]
8.0.201 [/usr/local/share/dotnet/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.11 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.12 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.13 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.15 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.16 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.18 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.23 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.11 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.12 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.13 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.15 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.16 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.18 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.23 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Other architectures found:
x64 [/usr/local/share/dotnet/x64]
Environment variables:
Not set
global.json file:
/Users/........./global.json
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
Anything else?
No response
The text was updated successfully, but these errors were encountered: