Agent MSI support #206

amitkanfer · 2023-10-26T15:09:01Z

Support basic functionality e2e (copy files and execute agent install from the target folder)
Support uninstall
Check return code from agent install and fail execution if code is different than 0
Support custom installation folder (EDIT: not sure this is needed. See comment here)
Implement the buildkite logic to trigger an MSI build whenever there's a new Agent version (similar to the logic we have with beats)
Cleanup all temp files after agent install is executed successfully
check installation with endpoint

hmnichols · 2023-10-26T15:50:58Z

Extremely excited to see this work, thank you!

gabriellandau · 2023-10-26T15:53:32Z

➕ Thank you @amitkanfer!

dliappis

Thanks for raising this @amitkanfer !

Took a first look at the Buildkite failure, left a question that might help us understand the problem.

src/Amalgamation.sln

gabriellandau · 2023-10-30T14:30:43Z

Is it possible to enroll Agent using this MSI, or is a subsequent command required to enroll Agent?

amitkanfer · 2023-10-30T14:42:29Z

Is it possible to enroll Agent using this MSI, or is a subsequent command required to enroll Agent?

Working on it right now, should be possible.

amitkanfer · 2023-10-31T08:09:49Z

I have a working version now.
The MSI:

copies all the contents of the zip file to c:\Program Files\Elastic\Beats\<version>\elastic-agent
Executes the process c:\Program Files\Elastic\Beats\<version>\elastic-agent\elastic-agent.exe with the arguments install -f <args> where args are the arguments that are passed to the MSI installer.

So for example, this works:
elastic-agent-8.10.4-windows-x86_64.msi INSTALLARGS=--url=<url> --enrollment-token=<token>

After the MSI finished copying all the files, it'll run the install command for the newly deployed elastic-agent.exe file. The installer will wait for the install command to finish and only then will exit. Logic can be found here. Worth noting that the process is ran w/o the command prompt window in the background.

What's missing:

Validate the functionality with infosec
Making sure we didn't change any logic for the beats MSI installers
Discuss how to tackle issues when the install command fails (due to network issues, token issues etc`). Should we rollback the installation? should we retry?
Make the CI happy (update: CI is happy now)

strawgate

Left a couple of comments

src/installer/BeatPackageCompiler/AgentCustomAction.cs

strawgate · 2023-10-31T14:53:03Z

src/installer/BeatPackageCompiler/AgentCustomAction.cs

+    public class AgentCustomAction
+    {
+        [CustomAction]
+        public static ActionResult MyAction(Session session)


My guess is we also need a custom action for uninstall?

yes we do. on it... not as straightforward as it seems, unfortunately.

strawgate · 2023-10-31T14:57:28Z

src/installer/BeatPackageCompiler/BeatPackageCompiler.cs

-            var textInfo = new CultureInfo("en-US", false).TextInfo;
-            var serviceDisplayName = $"{companyName} {textInfo.ToTitleCase(ap.TargetName)} {ap.SemVer}";
+            var beatsInstallPath =
+                $"[ProgramFiles{(ap.Is64Bit ? "64" : string.Empty)}Folder]" +


I think we only support 64-bit windows so this check is likely unnecessary?

Correct we only support 64-bit Windows.

Elastic Agent is not supported on 32-bit operating systems.

https://www.elastic.co/support/matrix

i can change this ofcourse... but note it's the original code that is also shared with the beats MSI, i tried not to change any logic for the beats MSI.

cmacknz · 2023-10-31T19:01:46Z

Make sure you can install Elastic Defend for an agent that was installed with this MSI.

cmacknz · 2023-10-31T19:11:49Z

A few more:

The MSI uninstall should call elastic-agent uninstall
We need to present errors from the elastic-agent install command back to the user in an obvious way. This is particularly important for Fleet enrollment where first time setup is likely to fail due to incorrect credentials, proxy configurations, etc.

nimarezainia · 2023-10-31T23:32:44Z

Would installation in a different directory path be something we need to test also?

amitkanfer · 2023-11-01T12:55:25Z

Updated the description with what's left to be implemented.

gabriellandau · 2023-11-01T14:16:55Z

Would installation in a different directory path be something we need to test also?

Do we need to support that on Windows? It adds complexity and risk. For example, if we allow users to install to user-writable directories, then our [un]install processes can end up writing/deleting user-controlled paths and we might unintentionally create LPEs and DoS. C:\Program Files is only-admin-writable by default, which prevents a variety of filesystem-related attacks such as this one.

amitkanfer · 2023-11-01T14:41:16Z

Do we need to support that on Windows? It adds complexity and risk. For example, if we allow users to install to user-writable directories, then our [un]install processes can end up writing/deleting user-controlled paths and we might unintentionally create LPEs and DoS. C:\Program Files is only-admin-writable by default, which prevents a variety of filesystem-related attacks such as this one.

funny you mention this. I was just implementing the uninstall flow, and nothing in the MSI package has information about the final destination of the install command (c:\Program Files\Elastic\Agent). I had to hard code it.

strawgate · 2023-11-01T16:06:51Z

The MSI way to change the directory is TARGETDIR https://learn.microsoft.com/en-us/windows/win32/msi/targetdir not sure if our current method bypasses that but refactoring around targetdir is probably the right way to get customizable install destination

amitkanfer · 2023-11-01T16:15:11Z

@strawgate i believe Nima was referring to where agent installs itself (which is a different path than where the MSI unpacks its contents). TARGETDIR is probably the right thing to change 99% of the time, however, as i explained here, the MSI unpacks the CAB files to TARGETDIR, but then calls agent install which again copies all the files to c:\Program Files\Elastic\Agent. I believe the latter is what Nima referred to.

btw, the latest version of the MSI cleans up the TARGETDIR folder after it successfully runs the agent install command.

amitkanfer · 2023-11-01T16:29:16Z

Another approach we could have taken is to unpack the contents of the MSI to the final folder (c:\Program Files\Elastic\Agent) and then register it as a service and then also call enroll. But after talking with @cmacknz we decided it'll be best to keep things simple and call install.

cmacknz · 2023-11-01T19:29:32Z

To change the target install path, we added the --base-path option to the install command. We could have the MSI set the value of that flag. It is slightly more complex than just unpacking agent in the new directory, some symlinks need to be set up and this is all encapsulated inside the install command today.

https://github.com/elastic/ingest-docs/blob/main/docs/en/ingest-management/commands.asciidoc#options-4

We disallowed changing the install path when using Elastic Defend, so at least in cases where someone is actively trying to use us for endpoint protection they aren't vulnerable to the arbitrary file deletion attacks Gabriel mentioned.

amitkanfer · 2023-11-19T20:37:34Z

Closing this PR in favor of: #212

amitkanfer added 2 commits October 26, 2023 15:47

First draft of creating an MSI for Elastic Agent

b07f66e

more fixes

6000717

amitkanfer requested a review from a team as a code owner October 26, 2023 15:09

'6'

b5e5f90

dliappis reviewed Oct 27, 2023

View reviewed changes

src/Amalgamation.sln Show resolved Hide resolved

'6'

c741e7a

amitkanfer added 3 commits October 31, 2023 09:45

Working version e2e

b87fc64

fix gitignore

c53470a

Add global.json

3da96ad

strawgate reviewed Oct 31, 2023

View reviewed changes

checking install command response code

5022ac9

amitkanfer added 2 commits November 1, 2023 17:20

Uninstall support

11d7a67

Add cleanup after successful agent install

4cdbe66

amitkanfer closed this Nov 19, 2023

amitkanfer mentioned this pull request Nov 19, 2023

Agent MSI support #212

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Agent MSI support #206

Agent MSI support #206

amitkanfer commented Oct 26, 2023 •

edited

Loading

hmnichols commented Oct 26, 2023

gabriellandau commented Oct 26, 2023

dliappis left a comment

gabriellandau commented Oct 30, 2023

amitkanfer commented Oct 30, 2023

amitkanfer commented Oct 31, 2023 •

edited

Loading

strawgate left a comment

strawgate Oct 31, 2023

amitkanfer Nov 1, 2023

strawgate Oct 31, 2023

gabriellandau Oct 31, 2023

amitkanfer Nov 1, 2023 •

edited

Loading

cmacknz commented Oct 31, 2023

cmacknz commented Oct 31, 2023

nimarezainia commented Oct 31, 2023

amitkanfer commented Nov 1, 2023

gabriellandau commented Nov 1, 2023 •

edited

Loading

amitkanfer commented Nov 1, 2023

strawgate commented Nov 1, 2023

amitkanfer commented Nov 1, 2023

amitkanfer commented Nov 1, 2023 •

edited

Loading

cmacknz commented Nov 1, 2023

amitkanfer commented Nov 19, 2023

Agent MSI support #206

Agent MSI support #206

Conversation

amitkanfer commented Oct 26, 2023 • edited Loading

hmnichols commented Oct 26, 2023

gabriellandau commented Oct 26, 2023

dliappis left a comment

Choose a reason for hiding this comment

gabriellandau commented Oct 30, 2023

amitkanfer commented Oct 30, 2023

amitkanfer commented Oct 31, 2023 • edited Loading

strawgate left a comment

Choose a reason for hiding this comment

strawgate Oct 31, 2023

Choose a reason for hiding this comment

amitkanfer Nov 1, 2023

Choose a reason for hiding this comment

strawgate Oct 31, 2023

Choose a reason for hiding this comment

gabriellandau Oct 31, 2023

Choose a reason for hiding this comment

amitkanfer Nov 1, 2023 • edited Loading

Choose a reason for hiding this comment

cmacknz commented Oct 31, 2023

cmacknz commented Oct 31, 2023

nimarezainia commented Oct 31, 2023

amitkanfer commented Nov 1, 2023

gabriellandau commented Nov 1, 2023 • edited Loading

amitkanfer commented Nov 1, 2023

strawgate commented Nov 1, 2023

amitkanfer commented Nov 1, 2023

amitkanfer commented Nov 1, 2023 • edited Loading

cmacknz commented Nov 1, 2023

amitkanfer commented Nov 19, 2023

amitkanfer commented Oct 26, 2023 •

edited

Loading

amitkanfer commented Oct 31, 2023 •

edited

Loading

amitkanfer Nov 1, 2023 •

edited

Loading

gabriellandau commented Nov 1, 2023 •

edited

Loading

amitkanfer commented Nov 1, 2023 •

edited

Loading