Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

CI: Failure to Checkout with Beta Package Added #2086

Closed
dadams39 opened this issue Jan 10, 2023 · 8 comments · Fixed by #2205
Closed

CI: Failure to Checkout with Beta Package Added #2086

dadams39 opened this issue Jan 10, 2023 · 8 comments · Fixed by #2205
Assignees
@dadams39
Labels
blocked Upstream item prevents completion bug Something isn't working help wanted Extra attention is needed

Comments

@dadams39
Copy link
Contributor

Failure during CI
This could be related to memory in the CI or another issue.

Blocks
@dadams39 dadams39 added bug Something isn't working help wanted Extra attention is needed labels Jan 10, 2023
@dadams39
Copy link
Contributor Author

Raw Logs:

2023-01-10T14:17:00.0847726Z ##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

Looks like the runner is running out of memory.

@dadams39
Copy link
Contributor Author

Blocker Upstream-Graph-SDK-129

@dadams39 dadams39 added the blocked Upstream item prevents completion label Jan 10, 2023
@dadams39
Copy link
Contributor Author

Temporarily, we can move in the two files from Beta. We will need to solve the problem of compilation with Beta to use the API for backups and restore.

@dadams39 dadams39 mentioned this issue Jan 10, 2023
Merged
3 tasks
@dadams39 dadams39 self-assigned this Jan 18, 2023
This was referenced Jan 18, 2023
Merged
Open
Closed
@dadams39
Copy link
Contributor Author

Suggestion from the comments is to increase the swap space on the runner:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Set Swap Space
        uses: pierotofy/set-swap-space@master
        with:
          swap-size-gb: 12

Need to verify if changes to the configuration will allow for the fix.

@dadams39 dadams39 mentioned this issue Jan 19, 2023
Closed
10 tasks
@dadams39
Copy link
Contributor Author

Setting the swap space to 15 gb resulted in:

Memory and swap:
               total        used        free      shared  buff/cache   available
Mem:           6.8Gi       526Mi       5.2Gi       7.0Mi       1.1Gi       6.0Gi
Swap:          4.0Gi          0B       4.0Gi

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G   0B   -2

@dadams39
Copy link
Contributor Author

Run with 12 GB of Swap.

Run pierotofy/set-swap-space@master
Run echo "Memory and swap:"
Memory and swap:
               total        used        free      shared  buff/cache   available
Mem:           [6](https://github.com/alcionai/corso/actions/runs/3960126507/jobs/6783853293#step:3:7).8Gi       548Mi       5.1Gi       7.0Mi       1.1Gi       5.9Gi
Swap:          4.0Gi          0B       4.0Gi

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G   0B   -2

Run export SWAP_FILE=$(swapon --show=NAME | tail -n 1)
Setting up swapspace version 1, size = 12 GiB (12884897792 bytes)
no label, UUID=45934725-a0c4-462d-a6a4-668924b10b9e
Run echo "Memory and swap:"
Memory and swap:
               total        used        free      shared  buff/cache   available
Mem:           6.8Gi       554Mi       5.1Gi       7.0Mi       1.1Gi       5.9Gi
Swap:           11Gi          0B        11Gi

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file  12G   0B   -2

Run Time 2h 2m

15s
ok github.com/alcionai/corso/src 0.031s [no tests to run]
ok github.com/alcionai/corso/src/cli 0.017s [no tests to run]
ok github.com/alcionai/corso/src/cli/backup 0.018s [no tests to run]
ok github.com/alcionai/corso/src/cli/config 0.017s [no tests to run]
ok github.com/alcionai/corso/src/cli/help 0.003s [no tests to run]
? github.com/alcionai/corso/src/cli/options [no test files]
ok github.com/alcionai/corso/src/cli/print 0.003s [no tests to run]
ok github.com/alcionai/corso/src/cli/repo 0.017s [no tests to run]
ok github.com/alcionai/corso/src/cli/restore 0.017s [no tests to run]
ok github.com/alcionai/corso/src/cli/utils 0.018s [no tests to run]
? github.com/alcionai/corso/src/cmd/factory [no test files]
? github.com/alcionai/corso/src/cmd/factory/impl [no test files]
? github.com/alcionai/corso/src/cmd/getM365 [no test files]
? github.com/alcionai/corso/src/cmd/mdgen [no test files]
? github.com/alcionai/corso/src/cmd/purge [no test files]

@dadams39 dadams39 mentioned this issue Jan 20, 2023
Merged
4 tasks
@dadams39
Copy link
Contributor Author

dadams39 commented Jan 20, 2023 •
edited
Loading

The above fix is not stable for multiple msgraph libraries. The first compilation was successful, but the following calls on linting and testing when attempting to access the cache.

Verified additional swap space is required for the beta.

@dadams39 dadams39 mentioned this issue Jan 21, 2023
Merged
3 tasks
@dadams39
Copy link
Contributor Author

dadams39 commented Jan 21, 2023 •
edited
Loading

Cache failure from run verified with only beta as main library. Beta library is not to be added to the Corso library.

An additional reason for not using the Beta library is that objects saved in v1.0 cannot be deserialized as model fields are current at the time they moved from Beta to v1.0. Additional changes to models.Contacts after it was added causes deserialization failures.

Approach: a Corso Connector for retrieving objects from Beta. See #2174

This was referenced Jan 21, 2023
Merged
Merged
aviator-app bot pushed a commit that referenced this issue Jan 31, 2023
GC: Beta Client Package (#2205)
3a52115 
## Description
To add needed features, Kiota-Generated Beta Connector created
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x] ✅ Yes, it's included


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #2086<issue>
* closes #2174 
* closes #2223

## Test Plan
Compilation and current CI tests will fail if the package does not align with current operational standards. 
- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Feb 3, 2023
GC: SharePoint: BackUp: Pages (#2178)
6e12885 
## Description
- Adds logic to retrieve  `SharePoint.Pages` from M365
- Anchor PR for `SharePoint.Pages` feature support. Restore Pipeline 
PR to remain in Draft to ensure PR Train is stable until the solution to #2174 is implemented. 
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x] ⛔ No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)
* related to #2173
* related to #2071
## Test Plan
- [x] ⚡ Unit test

NOTE: Tests will fail in CI due to complications with #2086.
aviator-app bot pushed a commit that referenced this issue Feb 6, 2023
GC: Restore: SharePoint: Page Logic (#2225)
d82b5ca 
## Description
Restore logic for restoring a SharePoint Page to M365 given a valid `[]byte`.  
Delete API also included

Tests included
<!-- Insert PR description-->

## Does this PR need a docs update or release note?
- [x] ⛔ No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to  #2169<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] ⚡ Unit test
Must be tested locally due to CI Library issues, See #2086.
Clean-up is handled within the tests.
aviator-app bot pushed a commit that referenced this issue Feb 8, 2023
GC: SharePoint: Backup: Abstract / Serialize (#2187)
3d244c9 
## Description
Changes address updates to `sharePoint.Collection.Populate()`. 
- SharePoint Collections support `Lists` and `Pages`. Drives are supported in OneDrive at this time. 
- List's serialize function is abstracted to support `Pages`. 

Collection needs to support List and Pages support. Additionally, needs to use a similar interface as in `exchange` to ensure retry and async collection population
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x] ✅ Yes, it's included


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to #2071 <issue>

## Test Plan
Test can be completed locally. Per Issue #2086, the Beta library interferes with overall caching. 
`TestSharePointPageCollection_Populate()` inspects changes
<!-- How will this be tested prior to merging.-->
- [x] ⚡ Unit test
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dadams39 dadams39

Labels
blocked Upstream item prevents completion bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

checkout-with-beta-package alcionai/corso
GC: Beta Client Package alcionai/corso
1 participant
@dadams39