Skip to content

Error with multiple FI in layer_output mode #8

@uditagarwal97

Description

@uditagarwal97

Here's the error log:

(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ python3 -c 'import keras; print(keras.__version__)'
2.4.3
(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ python3 cnn-mnist.py ./confFiles/sample.yaml ./result/ 100 1 > /dev/null
2021-10-22 18:27:29.528464: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
/home/uditg/UBC/TensorFI2/src/config.py:13: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  fiConf = yaml.load(fiConfs)
Traceback (most recent call last):
  File "cnn-mnist.py", line 79, in <module>
    res = tfi.inject(model=model, x_test=test_images[i:i+1], confFile=conf)
  File "/home/uditg/UBC/TensorFI2/src/tensorfi2.py", line 46, in __init__
    self.val = fiFunc(model, fiConf, **kwargs)
  File "/home/uditg/UBC/TensorFI2/src/tensorfi2.py", line 316, in layer_outputs
    pred = get_pred([fiLayerOutputs])
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 3792, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1605, in __call__
    return self._call_impl(args, kwargs)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1645, in _call_impl
    return self._call_flat(args, self.captured_inputs, cancellation_manager)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 598, in call
    ctx=ctx)
  File "/home/uditg/anaconda/envs/tfi2/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError:  In[0] is not a matrix. Instead it has shape [1,12,12,32]
         [[node dense/BiasAdd (defined at /home/uditg/UBC/TensorFI2/src/tensorfi2.py:315) ]] [Op:__inference_keras_scratch_graph_1104]

Function call stack:
keras_scratch_graph

(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ cat ./confFiles/sample.yaml 
Target: layer_outputs
Mode: multiple
Type: bitflips
Amount: 10
Bit: N
(tfi2) uditg@2403c486b851:~/UBC/TFI2_Original/TensorFI2/experiments/layer-outputs$ 

System config:

(tfi2) $ python3 -c 'import tensorflow as tf; print(tf.__version__)'                                                                    
2.2.0                                                                                                                                                                                                             
(tfi2) $ python3 -c 'import keras; print(keras.__version__)'                                                                            
2.4.3                                                                                                                                                                                                             
$ python --version                                                                                                               
Python 3.7.10                                                                                                                                                                                                     
$ python3 --version                                                                                                          
Python 3.7.10                                                                                                                                                                                                     
$ uname -a                                                                                                                    
Linux 2403c486b851 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux                                                                                               
$ cat /etc/os-release                                                                                                    
NAME="Ubuntu"                                                                                                                                                                                                     
VERSION="18.04.5 LTS (Bionic Beaver)"                                                                                                                                                                                                                                                                                                                                                                               
PRETTY_NAME="Ubuntu 18.04.5 LTS"                                                                                                                                                                                  
VERSION_ID="18.04"                                                                                                                                                                                                

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions