Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build CircleCI trop longue en général #1503

Closed
thbar opened this issue Feb 11, 2021 · 4 comments
Closed

Build CircleCI trop longue en général #1503

thbar opened this issue Feb 11, 2021 · 4 comments
Labels
dette technique Entretien & maintenance générale, nécessaire pour que le code reste de bonne qualité

Comments

@thbar
Copy link
Contributor

thbar commented Feb 11, 2021

En travaillant sur #1457, je remarque que la build est très longue, parfois tellement que le build PLT par exemple fait partir le tout en timeout:

Copying dialyxir_erlang-22.3.4.12_elixir-1.10.4.plt to dialyxir_erlang-22.3.4.12_elixir-1.10.4_deps-dev.plt
Looking up modules in dialyxir_erlang-22.3.4.12_elixir-1.10.4_deps-dev.plt
Checking 437 modules in dialyxir_erlang-22.3.4.12_elixir-1.10.4_deps-dev.plt
Adding 1983 modules to dialyxir_erlang-22.3.4.12_elixir-1.10.4_deps-dev.plt
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
Too long with no output (exceeded 10m0s): context deadline exceeded

De façon générale je me pose les questions suivantes:

  • Peut-on éviter une double compilation (dev + test) tout en conservant l'usage de dialyzer ?
  • Qu'est-ce qui est réellement mis en cache en pratique, et comment améliorer le cache en général ?
  • Si certains tests d'intégrations pourraient passer en test unitaire en changeant de stratégie

En filigrane, le temps de déploiement est un peu idem, car actuellement l'application est recompilée intégralement. On ne pourra pas changer ça à court terme, mais peut-être plus tard.

J'ouvre le ticket pour aller voir ce point plus tard.

@thbar
Copy link
Contributor Author

thbar commented Feb 11, 2021

Dans #1457 je mets déjà à jour dialyxir, même si je ne pense pas que ça va changer drastiquement vu le changelog.

@thbar
Copy link
Contributor Author

thbar commented Feb 11, 2021

Dans #1457 je fais également une mise en cache de la partie compilation après chaque étape (compilation dev, compilation test), ce qui permet de laisser plus de temps. J'ai également modifié la configuration dialyxir pour ne plus faire de travail en transitif sur les dépendances, simplement les dépendances directes, en attendant mieux.

@thbar
Copy link
Contributor Author

thbar commented Feb 12, 2021

Voir elixir-lang/elixir#9980 (comment) pour des points intéressants avec Elixir 11 (quand on aura mis à jour).

@thbar thbar added the dette technique Entretien & maintenance générale, nécessaire pour que le code reste de bonne qualité label Feb 16, 2021
thbar added a commit that referenced this issue Feb 18, 2021
* Start upgrading Phoenix

* Follow changelog advice

* Fix deprecation warning

* Fix deprecation warning

* Adapt code for deprecation (pubsub)

* Adapt layout rendering as recommended

* Upgrade phoenix_html_sanitizer

* Fix difference seen in PhoenixDiff

This is not required right now, but could be confusing to others (or myself) later.

* Upgrade phoenix_live_reload

* Upgrade gettext

* Update timex to remove compilation warnings

* Add git to the CircleCI build

* Use alpine-compatible git install command

* Add missing openssh-client package

* Try to work around CircleCI build problem

* Update eternal to fix deprecation warning

* Fix duplicate init_test_session/2 error

* Log where the git binary is

* Attempt to save the git binary

* Fix incantation to share the git binary

* Add quickfix for missing git binary

Until etalab/transport-ops#19 is handled.

* Fix major blocking point

Many tests would previously fail with the following error:

```
function Phoenix.Template.HTML.encode_to_iodata!/1 is undefined (module Phoenix.Template.HTML is not available)
```

After a fair bit of digging, I found that the default config today is different:

https://github.com/jayjun/phoenix/blob/master/lib/phoenix/template.ex#L223-L225

`Phoenix.HTML.Engine` should be used instead of `Phoenix.Template.HTML`.

I am removing the entry completely since:
- `html` will be handled fine with the default configuration
- `json` will leverage `Phoenix.json_library`, which is configured just above

* Add useful boilerplate for later debugging

* Remove fetch_flash since we already have fetch_live_flash

The default boilerplate does not use both, and fetch_live_flash includes the previous one (https://github.com/phoenixframework/phoenix_live_view/blob/v0.15.4/lib/phoenix_live_view/router.ex#L132).

* Remove unused resource

* Bump up Elixir dependency

* Remove TODO for credo

* Bump phoenix_live_view to latest

* Try to fix solution tests by updating the chrome container

* Update hound to latest

* Stop using dialyxir release candidate & adjust configuration

* Try to save some time during build debugging

The build fails at PLT and I want to debug that, but each build will also recompiled dev + test each time, so I'm setting intermediate snapshots to hopefully shave off 7 minutes per build.

* Fix deprecation warning

* Attempt to reduce time taken by PLT construction for now

* Add note

* Fix linter error

* Bump up yarn dependencies

* Make browser window size deterministic (fixes #1504)

* set chrome driver version to 86

As 88 make find_element function fail.

* Add useful doc

* Switch to dialyxir master version to get plt_local_path option

* Save PLT files to a well-defined place

* Ignore PLT files

* Attempt to optimize build time (#1503, #1505)

* Allow 20 minutes without STDOUT output for Dialyzer PLT construction

* Attempt to fix dev/test compilation caching inefficience

* Empty commit to trigger CI build

* Revert to app_tree

Co-authored-by: Francis <[email protected]>
@thbar
Copy link
Contributor Author

thbar commented Feb 18, 2021

La build a été optimisée dans #1457/#1505.

On pourra optimiser en passant à Elixir 11.

Je clôture pour l'instant, la suite sera dans #1509.

@thbar thbar closed this as completed Feb 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dette technique Entretien & maintenance générale, nécessaire pour que le code reste de bonne qualité
Projects
None yet
Development

No branches or pull requests

1 participant