No module named pyspark

I have installed toree to my jupyter environment (https://github.com/apache/incubator-toree) and written a piece of code that works with pyspark. Yarn starts properly and I can see the containers running in the queue,

When I run the code, I get the following error

Error from python worker:
/usr/local/bin/python2.7: No module named pyspark

the kernel is set-up as follows:

{
"language": "python",
"display_name": "Apache Toree - PySpark",
"env":

{ "__TOREE_SPARK_OPTS__": " --master yarn", "SPARK_HOME": "/usr/hdp/2.4.2.0-258/spark", "__TOREE_OPTS__": "", "DEFAULT_INTERPRETER": "PySpark", "PYTHONPATH": "/usr/hdp/2.4.2.0-258/spark/python:/usr/hdp/2.4.2.0-258/spark/python/lib/py4j-0.9-src.zip", "PYTHON_EXEC": "python", "PYTHONSTARTUP": "/usr/hdp/2.4.2.0-258/spark/python/pyspark/shell.py", "PYSPARK_PYTHON": "/usr/local/bin/python2.7", "PYSPARK_DRIVER_PYTHON": "/usr/local/bin/python2.7" }

,
"argv": [
"/usr/local/share/jupyter/kernels/apache_toree_pyspark/bin/run.sh",
"--profile",
"

{connection_file}

"
]
}