[Bug] buildx cannot build what docker build can build

[shizhMSFT]

Taking example of https://github.com/jupyter/docker-stacks/tree/master/all-spark-notebook.

all-spark-notebook can be built by normal docker build as

git clone https://github.com/jupyter/docker-stacks.git
cd docker-stacks/all-spark-notebook
docker build -t all .

However, it fails when using buildx:

git clone https://github.com/jupyter/docker-stacks.git
cd docker-stacks/all-spark-notebook
docker buildx build -t allx .

Here is the error message

$ docker buildx build -t allx .                                                    
[+] Building 276.2s (8/9)                                                                                                             
 => [internal] load build definition from Dockerfile                                                                              0.2s
 => => transferring dockerfile: 1.42kB                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                 0.3s
 => => transferring context: 66B                                                                                                  0.0s
 => [internal] load metadata for docker.io/jupyter/pyspark-notebook:latest                                                        0.0s
 => [1/6] FROM docker.io/jupyter/pyspark-notebook                                                                                 1.5s
 => => resolve docker.io/jupyter/pyspark-notebook:latest                                                                          0.0s
 => [2/6] RUN fix-permissions /usr/local/spark/R/lib                                                                              1.8s
 => [3/6] RUN apt-get update &&     apt-get install -y --no-install-recommends     fonts-dejavu     gfortran     gcc &&     rm   22.9s
 => [4/6] RUN conda install --quiet --yes     'r-base=3.5.1'     'r-irkernel=0.8*'     'r-ggplot2=3.1*'     'r-sparklyr=0.9*'   177.2s
 => ERROR [5/6] RUN pip install --no-cache-dir     https://dist.apache.org/repos/dist/release/incubator/toree/0.3.0-incubating/  72.1s
------
 > [5/6] RUN pip install --no-cache-dir     https://dist.apache.org/repos/dist/release/incubator/toree/0.3.0-incubating/toree-pip/toree
-0.3.0.tar.gz     &&     jupyter toree install --sys-prefix &&     rm -rf /home/jovyan/.local &&     fix-permissions /opt/conda &&    
fix-permissions /home/jovyan:
#8 1.456 Collecting https://dist.apache.org/repos/dist/release/incubator/toree/0.3.0-incubating/toree-pip/toree-0.3.0.tar.gz   
#8 2.195   Downloading https://dist.apache.org/repos/dist/release/incubator/toree/0.3.0-incubating/toree-pip/toree-0.3.0.tar.gz (25.9MB
)
#8 66.81 Requirement already satisfied: jupyter_core>=4.0 in /opt/conda/lib/python3.7/site-packages (from toree==0.3.0) (4.4.0)
#8 66.82 Requirement already satisfied: jupyter_client>=4.0 in /opt/conda/lib/python3.7/site-packages (from toree==0.3.0) (5.2.4)
#8 66.83 Requirement already satisfied: traitlets<5.0,>=4.0 in /opt/conda/lib/python3.7/site-packages (from toree==0.3.0) (4.3.2)
#8 66.84 Requirement already satisfied: tornado>=4.1 in /opt/conda/lib/python3.7/site-packages (from jupyter_client>=4.0->toree==0.3.0)
 (6.0.2)
#8 66.84 Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.7/site-packages (from jupyter_client>=4.0->toree
==0.3.0) (2.8.0) 
#8 66.85 Requirement already satisfied: pyzmq>=13 in /opt/conda/lib/python3.7/site-packages (from jupyter_client>=4.0->toree==0.3.0) (1
8.0.1)  
#8 66.85 Requirement already satisfied: ipython_genutils in /opt/conda/lib/python3.7/site-packages (from traitlets<5.0,>=4.0->toree==0.
3.0) (0.2.0)   
#8 66.85 Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from traitlets<5.0,>=4.0->toree==0.3.0) (1.12.0)
#8 66.85 Requirement already satisfied: decorator in /opt/conda/lib/python3.7/site-packages (from traitlets<5.0,>=4.0->toree==0.3.0) (4
.4.0)   
#8 66.85 Building wheels for collected packages: toree   
#8 66.86   Building wheel for toree (setup.py): started  
#8 68.55   Building wheel for toree (setup.py): finished with status 'done'   
#8 68.55   Stored in directory: /tmp/pip-ephem-wheel-cache-b297gkpf/wheels/1c/fe/6a/1a8d5d7d0274ccd5c160f3e2ef9477f3b071b7f9bb0ce6c96a
#8 68.82 Successfully built toree   
#8 69.19 Installing collected packages: toree
#8 69.29 Successfully installed toree-0.3.0
#8 69.69 [ToreeInstall] Installing Apache Toree version 0.3.0   
#8 69.69 [ToreeInstall] 
#8 69.69 Apache Toree is an effort undergoing incubation at the Apache Software 
#8 69.69 Foundation (ASF), sponsored by the Apache Incubator PMC. 
#8 69.69
#8 69.69 Incubation is required of all newly accepted projects until a further review
#8 69.69 indicates that the infrastructure, communications, and decision making process
#8 69.69 have stabilized in a manner consistent with other successful ASF projects.  
#8 69.69
#8 69.69 While incubation status is not necessarily a reflection of the completeness 
#8 69.69 or stability of the code, it does indicate that the project has yet to be   
#8 69.69 fully endorsed by the ASF. 
#8 69.69 [ToreeInstall] Creating kernel Scala
#8 69.69 Traceback (most recent call last):
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 528, in get  
#8 69.69     value = obj._trait_values[self.name] 
#8 69.69 KeyError: 'kernel_spec_manager'   
#8 69.69
#8 69.69 During handling of the above exception, another exception occurred:  
#8 69.69
#8 69.69 Traceback (most recent call last):
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 528, in get  
#8 69.69     value = obj._trait_values[self.name] 
#8 69.69 KeyError: 'data_dir'
#8 69.69
#8 69.69 During handling of the above exception, another exception occurred:  
#8 69.69
#8 69.69 Traceback (most recent call last):
#8 69.69   File "/opt/conda/bin/jupyter-toree", line 10, in <module>   
#8 69.69     sys.exit(main())
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/toree/toreeapp.py", line 165, in main      
#8 69.69     ToreeApp.launch_instance()    
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 658, in launch_instance  
#8 69.69     app.start()     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/toree/toreeapp.py", line 162, in start     
#8 69.69     return self.subapp.start()    
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/toree/toreeapp.py", line 125, in start     
#8 69.69     install_dir = self.kernel_spec_manager.install_kernel_spec(self.sourcedir,     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 556, in __get__     
#8 69.69     return self.get(obj, cls)     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 535, in get  
#8 69.69     value = self._validate(obj, dynamic_default())     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/jupyter_client/kernelspecapp.py", line 87, in _kernel_spec_manager_default    
#8 69.69     return KernelSpecManager(data_dir=self.data_dir)   
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 556, in __get__     
#8 69.69     return self.get(obj, cls)     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 535, in get  
#8 69.69     value = self._validate(obj, dynamic_default())     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/jupyter_core/application.py", line 93, in _data_dir_default     
#8 69.69     ensure_dir_exists(d, mode=0o700)     
#8 69.69   File "/opt/conda/lib/python3.7/site-packages/jupyter_core/utils/__init__.py", line 13, in ensure_dir_exists  
#8 69.69     os.makedirs(path, mode=mode)  
#8 69.69   File "/opt/conda/lib/python3.7/os.py", line 211, in makedirs
#8 69.69     makedirs(head, exist_ok=exist_ok)    
#8 69.69   File "/opt/conda/lib/python3.7/os.py", line 211, in makedirs
#8 69.69     makedirs(head, exist_ok=exist_ok)    
#8 69.69   File "/opt/conda/lib/python3.7/os.py", line 221, in makedirs
#8 69.69     mkdir(name, mode)      
#8 69.69 PermissionError: [Errno 13] Permission denied: '/home/jovyan/.local' 
------  
failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c pip install --no-cache-dir     https://dist.apach
e.org/repos/dist/release/incubator/toree/0.3.0-incubating/toree-pip/toree-0.3.0.tar.gz     &&     jupyter toree install --sys-prefix &&
     rm -rf /home/$NB_USER/.local &&     fix-permissions $CONDA_DIR &&     fix-permissions /home/$NB_USER]: exit code: 1

Here is the docker version

$ docker version
Client:
 Version:           19.03.0-beta3
 API version:       1.40
 Go version:        go1.12.4
 Git commit:        c55e026
 Built:             Thu Apr 25 02:58:59 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.0-beta3
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.4
  Git commit:       c55e026
  Built:            Thu Apr 25 02:57:32 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.5
  GitCommit:        bb71b10fd8f58240ca47fbb579b9d1028eea7c84
 runc:
  Version:          1.0.0-rc6+dev
  GitCommit:        2b18fe1d885ee5083ef9f0838fee39b62d653e30
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
$ docker buildx version
github.com/docker/buildx  .m

[tonistiigi]

This was fixed in moby/buildkit#1002 . Should already work with docker-container driver, and with docker driver after upgrading to beta4.

[shizhMSFT]

It works now with beta4.

/cc @northtyphoon