Dash Fashion MNIST Explorer does not deploy

[xhluca]

Seems like dash-fashion-mnist-explorer does not deploy on circleci. Will deployignore the app for now but we should investigate it later.

Show circleci failure logs

====> Deploying dash-fashion-mnist-explorer
      dashr: false
      dash python version: dash==1.7.0
      exists: true
      remote-sha: 
====> Initializing repository
====> Ensuring code is up to date
      Copying updated app source
      Python app detected, injecting common python-specific files
====> Deploying
Counting objects: 34, done.
Delta compression using up to 36 threads.
Compressing objects: 100% (33/33), done.
Writing objects: 100% (34/34), 32.66 MiB | 2.94 MiB/s, done.
Total 34 (delta 0), reused 0 (delta 0)
remote: -----> DDS disk space information        
remote: Filesystem      Size  Used Avail Use% Mounted on        
remote: overlay         2.0T  1.5T  512G  75% /        
remote: /dev/sdc        206G  117G   79G  60% /data        
remote: -----> DASH_APP_NAME => dash-fashion-mnist-explorer        
remote: -----> DASH_DOMAIN_BASE => dash-gallery.plotly.host        
remote: -----> DASH_PATH_ROUTING => 1        
remote: -----> DASH_STREAMBED_DIRECT_IP => 172.17.0.1        
remote: -----> DASH_LOGOUT_URL => /Manager/api/logout        
remote: -----> SCRIPT_NAME => /dash-fashion-mnist-explorer        
remote: -----> Cleaning up...        
remote: -----> Building dash-fashion-mnist-explorer from herokuish...        
remote: rehash: warning: skipping ca-certificates.pem,it does not contain exactly one certificate or CRL        
remote: 1 added, 0 removed; done.        
remote: Running hooks in /etc/ca-certificates/update.d...        
remote:         
remote: updates of cacerts keystore disabled.        
remote: done.        
remote: -----> Adding BUILD_ENV to build environment...        
remote:        -----> Python app detected        
remote: -----> Installing pip        
remote: -----> Installing requirements with pip        
remote:        Collecting setuptools>=41.0.0 (from tensorboard<1.15.0,>=1.14.0->tensorflow->-r /tmp/build/requirements.txt (line 5))        
remote:        Downloading https://files.pythonhosted.org/packages/9e/d4/b99a960314121a003e9f39c61dfde01a1010bb47661e193a7722f7f32d52/setuptools-54.2.0-py3-none-any.whl (785kB)        
remote:        Installing collected packages: setuptools        
remote:        Found existing installation: setuptools 39.0.1        
remote:        Uninstalling setuptools-39.0.1:        
remote:        Successfully uninstalled setuptools-39.0.1        
remote:        Successfully installed setuptools-54.2.0        
remote:        -----> Discovering process types        
remote:        Procfile declares types -> web        
remote: -----> Releasing dash-fashion-mnist-explorer (dokku/dash-fashion-mnist-explorer:latest)...        
remote: -----> Deploying dash-fashion-mnist-explorer (dokku/dash-fashion-mnist-explorer:latest)...        
remote:  !     Predeploy command declared: 'python predeploy.py'        
remote: -----> App Procfile file found (/home/dokku/dash-fashion-mnist-explorer/DOKKU_PROCFILE)        
remote:        DOKKU_SCALE declares scale -> web=1        
remote: =====> Processing deployment checks        
remote:        No CHECKS file found. Simple container checks will be performed.        
remote:        For more efficient zero downtime deployments, create a CHECKS file. See http://dokku.viewdocs.io/dokku/deployment/zero-downtime-deploys/ for examples        
remote: -----> Attempting pre-flight checks (web.1)        
remote:        Waiting for 10 seconds ...        
remote: 3be80fce7b898273adbf023137d00167fcf63f2e82ba15a108a0f760089f2db6        
remote: App container failed to start!!        
remote: =====> dash-fashion-mnist-explorer web container output:        
remote:        [2021-03-31 21:32:51 +0000] [12] [INFO] Starting gunicorn 19.9.0        
remote:        [2021-03-31 21:32:51 +0000] [12] [INFO] Listening at: http://0.0.0.0:5000 (12)        
remote:        [2021-03-31 21:32:51 +0000] [12] [INFO] Using worker: sync        
remote:        [2021-03-31 21:32:51 +0000] [189] [INFO] Booting worker with pid: 189        
remote:        Using TensorFlow backend.        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint8 = np.dtype([("qint8", np.int8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint8 = np.dtype([("quint8", np.uint8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint16 = np.dtype([("qint16", np.int16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint16 = np.dtype([("quint16", np.uint16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint32 = np.dtype([("qint32", np.int32, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          np_resource = np.dtype([("resource", np.ubyte, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint8 = np.dtype([("qint8", np.int8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint8 = np.dtype([("quint8", np.uint8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint16 = np.dtype([("qint16", np.int16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint16 = np.dtype([("quint16", np.uint16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint32 = np.dtype([("qint32", np.int32, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          np_resource = np.dtype([("resource", np.ubyte, 1)])        
remote:        [2021-03-31 21:33:01 +0000] [189] [ERROR] Exception in worker process        
remote:        Traceback (most recent call last):        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker        
remote:            worker.init_process()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process        
remote:            self.load_wsgi()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi        
remote:            self.wsgi = self.app.wsgi()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi        
remote:            self.callable = self.load()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load        
remote:            return self.load_wsgiapp()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp        
remote:            return util.import_app(self.app_uri)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app        
remote:            __import__(module)        
remote:          File "/app/app.py", line 30, in <module>        
remote:            model = load_model("trained_data/fashion_mnist_cnn.h5")        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 492, in load_wrapper        
remote:            return load_function(*args, **kwargs)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 584, in load_model        
remote:            model = _deserialize_model(h5dict, custom_objects, compile)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 273, in _deserialize_model        
remote:            model_config = json.loads(model_config.decode('utf-8'))        
remote:        AttributeError: 'str' object has no attribute 'decode'        
remote:        [2021-03-31 21:33:01 +0000] [189] [INFO] Worker exiting (pid: 189)        
remote:        [2021-03-31 21:33:01 +0000] [12] [INFO] Shutting down: Master        
remote:        [2021-03-31 21:33:01 +0000] [12] [INFO] Reason: Worker failed to boot.        
remote:        [2021-03-31 21:33:04 +0000] [15] [INFO] Starting gunicorn 19.9.0        
remote:        [2021-03-31 21:33:04 +0000] [15] [INFO] Listening at: http://0.0.0.0:5000 (15)        
remote:        [2021-03-31 21:33:04 +0000] [15] [INFO] Using worker: sync        
remote:        [2021-03-31 21:33:04 +0000] [191] [INFO] Booting worker with pid: 191        
remote:        Using TensorFlow backend.        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint8 = np.dtype([("qint8", np.int8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint8 = np.dtype([("quint8", np.uint8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint16 = np.dtype([("qint16", np.int16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint16 = np.dtype([("quint16", np.uint16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint32 = np.dtype([("qint32", np.int32, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          np_resource = np.dtype([("resource", np.ubyte, 1)])        
remote:        [2021-03-31 21:33:05 +0000] [15] [INFO] Handling signal: term        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint8 = np.dtype([("qint8", np.int8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint8 = np.dtype([("quint8", np.uint8, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint16 = np.dtype([("qint16", np.int16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_quint16 = np.dtype([("quint16", np.uint16, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          _np_qint32 = np.dtype([("qint32", np.int32, 1)])        
remote:        /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.        
remote:          np_resource = np.dtype([("resource", np.ubyte, 1)])        
remote:        [2021-03-31 21:33:09 +0000] [191] [ERROR] Exception in worker process        
remote:        Traceback (most recent call last):        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker        
remote:            worker.init_process()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process        
remote:            self.load_wsgi()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi        
remote:            self.wsgi = self.app.wsgi()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi        
remote:            self.callable = self.load()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load        
remote:            return self.load_wsgiapp()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp        
remote:            return util.import_app(self.app_uri)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app        
remote:            __import__(module)        
remote:          File "/app/app.py", line 30, in <module>        
remote:            model = load_model("trained_data/fashion_mnist_cnn.h5")        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 492, in load_wrapper        
remote:            return load_function(*args, **kwargs)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 584, in load_model        
remote:            model = _deserialize_model(h5dict, custom_objects, compile)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 273, in _deserialize_model        
remote:            model_config = json.loads(model_config.decode('utf-8'))        
remote:        AttributeError: 'str' object has no attribute 'decode'        
remote:        [2021-03-31 21:33:09 +0000] [191] [INFO] Worker exiting (pid: 191)        
remote:        Traceback (most recent call last):        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 225, in run        
remote:            handler()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 260, in handle_term        
remote:            raise StopIteration        
remote:        StopIteration        
remote:        During handling of the above exception, another exception occurred:        
remote:        Traceback (most recent call last):        
remote:          File "/app/.heroku/python/bin/gunicorn", line 11, in <module>        
remote:            sys.exit(run())        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run        
remote:            WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run        
remote:            super(Application, self).run()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run        
remote:            Arbiter(self).run()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 228, in run        
remote:            self.halt()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt        
remote:            self.stop()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop        
remote:            time.sleep(0.1)        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld        
remote:            self.reap_workers()        
remote:          File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers        
remote:            raise HaltServer(reason, self.WORKER_BOOT_ERROR)        
remote:        gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>        
remote: =====> end dash-fashion-mnist-explorer web container output        
To ********************************/GIT/dash-fashion-mnist-explorer
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '********************************/GIT/dash-fashion-mnist-explorer'

Exited with code exit status 1
CircleCI received exit code 1

[xhluca]

Seems like the issue still persists when run locally. The issue seems to stem from the version mismatch between tensorflow and the Keras-2.3.1. Unfortunately, I tried downgrading tensorflow to 1.x and change keras to tensorflow.keras, but the app still does not work correctly. I will keep it ignored until further investigation.