Skip to content

Commit

Permalink
feat: initial import
Browse files Browse the repository at this point in the history
  • Loading branch information
benoitf committed Nov 18, 2021
0 parents commit 1ab5da9
Show file tree
Hide file tree
Showing 12 changed files with 502 additions and 0 deletions.
7 changes: 7 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.git
code/.build/**
code/out**
code/node_modules/**
Dockerfile
vscode-reh-*
tools
61 changes: 61 additions & 0 deletions .github/workflows/pr-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#
# Copyright (c) 2021 Red Hat, Inc.
# This program and the accompanying materials are made
# available under the terms of the Eclipse Public License 2.0
# which is available at https://www.eclipse.org/legal/epl-2.0/
#
# SPDX-License-Identifier: EPL-2.0
#

name: pr-check

# Trigger the workflow on pull request
on: [pull_request]

jobs:

build:
name: build
strategy:
fail-fast: false
matrix:
dist: [ 'musl', 'libc' ]
arch: ['amd64']
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Docker Build
run: |
docker buildx build --memory-swap -1 --memory 10g --platform linux/${{matrix.arch}} -f build/dockerfiles/linux-${{matrix.dist}}.Dockerfile --load -t linux-${{matrix.dist}}-${{matrix.arch}} .
- name: Upload image
uses: ishworkh/docker-image-artifact-upload@v1
with:
image: "linux-${{matrix.dist}}-${{matrix.arch}}"
assemble:
name: assemble
needs: build
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Checkout
uses: actions/checkout@v2
- name: Download linux-libc-amd64 image
uses: ishworkh/docker-image-artifact-download@v1
with:
image: "linux-libc-amd64"
- name: Download linux-musl image
uses: ishworkh/docker-image-artifact-download@v1
with:
image: "linux-musl-amd64"
- name: Display docker images
run: |
docker images
- name: Docker Build
run: |
docker buildx build --platform linux/amd64 -f build/dockerfiles/assembly.Dockerfile --load -t assembly-pr .
68 changes: 68 additions & 0 deletions .github/workflows/publish-next.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#
# Copyright (c) 2021 Red Hat, Inc.
# This program and the accompanying materials are made
# available under the terms of the Eclipse Public License 2.0
# which is available at https://www.eclipse.org/legal/epl-2.0/
#
# SPDX-License-Identifier: EPL-2.0
#

name: publish-next

on:
push:
branches: [ main ]

jobs:

build:
name: build
strategy:
fail-fast: false
matrix:
dist: [ 'musl', 'libc' ]
arch: ['amd64']
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Docker Build
run: |
docker buildx build --memory-swap -1 --memory 10g --platform linux/${{matrix.arch}} -f build/dockerfiles/linux-${{matrix.dist}}.Dockerfile --load -t linux-${{matrix.dist}}-${{matrix.arch}} .
- name: Upload image
uses: ishworkh/docker-image-artifact-upload@v1
with:
image: "linux-${{matrix.dist}}-${{matrix.arch}}"
assemble:
name: assemble
needs: build
runs-on: ubuntu-20.04
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Checkout
uses: actions/checkout@v2
- name: Download linux-libc-amd64 image
uses: ishworkh/docker-image-artifact-download@v1
with:
image: "linux-libc-amd64"
- name: Download linux-musl image
uses: ishworkh/docker-image-artifact-download@v1
with:
image: "linux-musl-amd64"
- name: Display docker images
run: |
docker images
- name: Login to Quay.io
uses: docker/login-action@v1
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
- name: Docker Build
run: |
docker buildx build --platform linux/amd64 -f build/dockerfiles/assembly.Dockerfile --push -t quay.io/che-incubator/che-code:next .
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
vscode-reh-web-*
node_modules

75 changes: 75 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# che-code

Deploy `Code-OSS` on a kubernetes cluster and connect with your Browser.

This repository is hosting the changes to have the `Code-OSS` running inside a Browser and connecting to a remote HTTP(s) server (instead of using Desktop mode).

The terminal is aware of the running Pod. Then, you can open terminals in every container of the running pod (if the containers have shell access).

Upstream `Code-OSS` is stored using Git [subtree](https://git-scm.com/book/en/v2/Git-Tools-Advanced-Merging#_subtree_merge). It means that if you're not interested in updating/rebasing upstream code you don't need to setup anything else unlike git submodules. This repository is self-contained.

## Development pre-requisites
- NodeJS version used by `Code-OSS` (Exact version can be find inside https://github.com/microsoft/vscode/blob/main/remote/.yarnrc with target property)
- Yarn v1.x

## Directories Layout

- `code` contains the upstream content (subtree) + changes required to have Code running in a remote server.
- `deploy/helm`: Helm templates to deploy it on a k8s cluster
- `Dockerfile` is for building the container.
- `package.json` holds some top-level scripts that you can find also in the `code` folder.

## Development mode

1. Fetch dependencies with `yarn` command
2. Compile and watch folders: `yarn run watch`
3. Run the server (another terminal for example): `yarn run server`


## Updates and branches

This repository has a main branch being rebased on the main remote branch of `Code-OSS`.
Then, for each stable version of `Code-OSS`there is a matching branch.
For example remote `release/1.60` is handled locally as a `1.60.x` branch.

### Pulling/Diff against new Code OSS version - useful commands

Add the `Code-OSS` remote by using for example the following command:

```bash
$ git remote add upstream-code https://github.com/microsoft/vscode
```

#### Pull changes from the remote Code

For a release branch:

```bash
$ git subtree pull --prefix code upstream-code release/1.60 --squash
```

For the main branch:

```bash
$ git subtree pull --prefix code upstream-code main --squash
```

#### Check the diff between local and remote

For a release branch:

```bash
$ git diff upstream-code/release/1.60 1.60.x:code
```

For a main branch:

```bash
$ git diff upstream-code/main main:code
```


#### get the sha1 of the current subtree
```bash
$ git log -1 --grep="^git-subtree-dir: code/*\$" --no-show-signature | grep "git-subtree-split:" | tail -n 1 | cut -d ':' -f 2 | tr -d ' '
```
30 changes: 30 additions & 0 deletions build/dockerfiles/assembly.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Grab content from previously build images
FROM linux-libc-amd64 as linux-libc-content
FROM linux-musl-amd64 as linux-musl-content

FROM registry.access.redhat.com/ubi8/ubi:8.5-200 AS ubi-builder
RUN mkdir -p /mnt/rootfs
RUN yum install --installroot /mnt/rootfs brotli libstdc++ coreutils glibc-minimal-langpack --releasever 8 --setopt install_weak_deps=false --nodocs -y && yum --installroot /mnt/rootfs clean all
RUN rm -rf /mnt/rootfs/var/cache/* /mnt/rootfs/var/log/dnf* /mnt/rootfs/var/log/yum.*

WORKDIR /mnt/rootfs

COPY --from=linux-musl-content --chown=0:0 /checode-linux-musl /mnt/rootfs/checode-linux-musl
COPY --from=linux-libc-content --chown=0:0 /checode-linux-libc /mnt/rootfs/checode-linux-libc

RUN mkdir -p /mnt/rootfs/projects && mkdir -p /mnt/rootfs/home/che
RUN cat /mnt/rootfs/etc/passwd | sed s#root:x.*#root:x:\${USER_ID}:\${GROUP_ID}::\${HOME}:/bin/bash#g > /mnt/rootfs/home/che/.passwd.template \
&& cat /mnt/rootfs/etc/group | sed s#root:x:0:#root:x:0:0,\${USER_ID}:#g > /mnt/rootfs/home/che/.group.template
RUN for f in "/mnt/rootfs/home/che" "/mnt/rootfs/etc/passwd" "/mnt/rootfs/etc/group" "/mnt/rootfs/projects" ; do\
chgrp -R 0 ${f} && \
chmod -R g+rwX ${f}; \
done

COPY --chmod=755 /build/scripts/*.sh /mnt/rootfs/

# Create all-in-one image
FROM scratch
COPY --from=ubi-builder /mnt/rootfs/ /
ENV HOME=/home/che
USER 1001
ENTRYPOINT /entrypoint.sh
93 changes: 93 additions & 0 deletions build/dockerfiles/linux-libc.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# https://access.redhat.com/containers/?tab=tags#/registry.access.redhat.com/ubi8/nodejs-14
FROM registry.access.redhat.com/ubi8/nodejs-14:1-50 as linux-libc-builder

USER root
# Install libsecret-devel on s390x and ppc64le for keytar build (binary included in npm package for x86)
RUN { if [[ $(uname -m) == "s390x" ]]; then LIBSECRET="\
https://rpmfind.net/linux/fedora-secondary/releases/34/Everything/s390x/os/Packages/l/libsecret-0.20.4-2.fc34.s390x.rpm \
https://rpmfind.net/linux/fedora-secondary/releases/34/Everything/s390x/os/Packages/l/libsecret-devel-0.20.4-2.fc34.s390x.rpm"; \
elif [[ $(uname -m) == "ppc64le" ]]; then LIBSECRET="\
libsecret \
https://rpmfind.net/linux/centos/8-stream/BaseOS/ppc64le/os/Packages/libsecret-devel-0.18.6-1.el8.ppc64le.rpm"; \
elif [[ $(uname -m) == "x86_64" ]]; then LIBSECRET="\
https://rpmfind.net/linux/centos/8-stream/BaseOS/x86_64/os/Packages/libsecret-devel-0.18.6-1.el8.x86_64.rpm \
libsecret"; \
elif [[ $(uname -m) == "aarch64" ]]; then LIBSECRET="\
https://rpmfind.net/linux/centos/8-stream/BaseOS/aarch64/os/Packages/libsecret-devel-0.18.6-1.el8.aarch64.rpm \
libsecret"; \
else \
LIBSECRET=""; echo "Warning: arch $(uname -m) not supported"; \
fi; } \
&& { if [[ $(uname -m) == "x86_64" ]]; then LIBKEYBOARD="\
https://rpmfind.net/linux/centos/8-stream/AppStream/x86_64/os/Packages/libxkbfile-1.1.0-1.el8.x86_64.rpm \
https://rpmfind.net/linux/centos/8-stream/PowerTools/x86_64/os/Packages/libxkbfile-devel-1.1.0-1.el8.x86_64.rpm"; \
elif [[ $(uname -m) == "aarch64" ]]; then LIBKEYBOARD="\
https://rpmfind.net/linux/centos/8-stream/AppStream/aarch64/os/Packages/libxkbfile-1.1.0-1.el8.aarch64.rpm \
https://rpmfind.net/linux/centos/8-stream/PowerTools/aarch64/os/Packages/libxkbfile-devel-1.1.0-1.el8.aarch64.rpm"; \
else \
LIBKEYBOARD=""; echo "Warning: arch $(uname -m) not supported"; \
fi; } \
&& yum install -y $LIBSECRET $LIBKEYBOARD curl make cmake gcc gcc-c++ python2 git git-core-doc openssh less libX11-devel libxkbcommon bash tar gzip rsync patch \
&& yum -y clean all && rm -rf /var/cache/yum \
&& npm install -g [email protected]
COPY code /checode-compilation
WORKDIR /checode-compilation
ENV ELECTRON_SKIP_BINARY_DOWNLOAD=1 \
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1

# Initialize a git repository for code build tools
RUN git init .

# change network timeout (slow using multi-arch build)
RUN yarn config set network-timeout 600000 -g

# Grab dependencies (and force to rebuild them)
RUN yarn install --force

# Compile
RUN NODE_ARCH=$(echo "console.log(process.arch)" | node) \
&& NODE_VERSION=$(cat /checode-compilation/remote/.yarnrc | grep target | cut -d ' ' -f 2 | tr -d '"') \
# cache node from this image to avoid to grab it from within the build
&& mkdir -p /checode-compilation/.build/node/v${NODE_VERSION}/linux-${NODE_ARCH} \
&& echo "caching /checode-compilation/.build/node/v${NODE_VERSION}/linux-${NODE_ARCH}/node" \
&& cp /usr/bin/node /checode-compilation/.build/node/v${NODE_VERSION}/linux-${NODE_ARCH}/node \
&& NODE_OPTIONS="--max_old_space_size=8500" ./node_modules/.bin/gulp vscode-reh-web-linux-${NODE_ARCH}-min \
&& cp -r ../vscode-reh-web-linux-${NODE_ARCH} /checode

RUN chmod a+x /checode/out/vs/server/main.js \
&& chgrp -R 0 /checode && chmod -R g+rwX /checode

### Testing

# Compile test suites
# https://github.com/microsoft/vscode/blob/cdde5bedbf3ed88f93b5090bb3ed9ef2deb7a1b4/test/integration/browser/README.md#compile
RUN [[ $(uname -m) == "x86_64" ]] && yarn --cwd test/smoke compile && yarn --cwd test/integration/browser compile

# install test dependencies
ENV PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=0
RUN [[ $(uname -m) == "x86_64" ]] && yarn playwright-install
# Install procps to manage to kill processes and centos stream repository
RUN [[ $(uname -m) == "x86_64" ]] && \
ARCH=$(uname -m) && \
yum install --nobest -y procps \
https://rpmfind.net/linux/epel/8/Everything/${ARCH}/Packages/e/epel-release-8-13.el8.noarch.rpm \
http://mirror.centos.org/centos/8-stream/BaseOS/${ARCH}/os/Packages/centos-gpg-keys-8-3.el8.noarch.rpm \
http://mirror.centos.org/centos/8-stream/BaseOS/${ARCH}/os/Packages/centos-stream-repos-8-3.el8.noarch.rpm

RUN [[ $(uname -m) == "x86_64" ]] && yum install -y chromium && \
rm /opt/app-root/src/.cache/ms-playwright/chromium-930007/chrome-linux/chrome && \
ln -s /usr/bin/chromium-browser /opt/app-root/src/.cache/ms-playwright/chromium-930007/chrome-linux/chrome

# Run integration tests (Browser)
RUN [[ $(uname -m) == "x86_64" ]] && NODE_ARCH=$(echo "console.log(process.arch)" | node) \
VSCODE_REMOTE_SERVER_PATH="$(pwd)/../vscode-reh-web-linux-${NODE_ARCH}" \
./resources/server/test/test-web-integration.sh --browser chromium

# Run smoke tests (Browser)
RUN [[ $(uname -m) == "x86_64" ]] && NODE_ARCH=$(echo "console.log(process.arch)" | node) \
VSCODE_REMOTE_SERVER_PATH="$(pwd)/../vscode-reh-web-linux-${NODE_ARCH}" \
yarn smoketest-no-compile --web --headless --electronArgs="--disable-dev-shm-usage --use-gl=swiftshader"

# Store the content of the result
FROM scratch as linux-libc-content
COPY --from=linux-libc-builder /checode /checode-linux-libc
Loading

0 comments on commit 1ab5da9

Please sign in to comment.