Clare S. Y. Huang Data Scientist | Atmospheric Dynamicist

I love to share what I've learnt with others. Check out my blog posts and notes about my academic research, as well as technical solutions on software engineering and data science challenges.


Resources on Python Packaging

On my way figuring out how to properly write unit test for my python package, I have come across the useful pages below:

Unit tests are worth the time writing to make sure your package works as you expected. I also found some commercial packages using unit tests as sample script for user to refer to (e.g. AllenNLP).

Setting up a Dash App on PythonAnywhere

After opening an account on pythonanywhere, go to the Web tab and select Add a new web app.

When prompted to select a Python Web framework, choose Flask.

Choose your python version. Here, I am choosing Python 3.6 (Flask 0.12).

Enter a path for a Python file I wish to hold my Dash app. I entered:

/home/username/mysite/dashing_demo_app.py

Put the script of your Dash app in dashing_demo_app.py. You can use the script in the sample file dashing_demo_app.py provided on the GitHub repo of pythonanywhere’s staff.

Next I have to set up a virtual environment that the app is running in. I am using the requirements3.6.txt provided in the above GitHub repo.

Go to the Files tab to create requirements3.6.txt in your home directory. Then, go to the Consoles tab to start a new bash session. Create a virtual environment dashappenv with the following command in the home directory:

mkvirtualenv dashappenv --python=/usr/bin/python3.6
pip install -r requirements3.6.txt

Then, go to the Web tab and enter under Virtualenv the path of your virtual environment:

/home/username/.virtualenvs/dashappenv

Lastly, modify your WSGI file. Instead of

from dashing_demo_app import app as application

provided, enter

from dashing_demo_app import app
application = app.server

to import your app.

It’s all done. Go to Web to reload your app. You can then click the URL of your webapp and see it running. :) Here is the sample webapp I built based on the example in Dash tutorial.

Published on Science!

Our paper, Nakmaura and Huang (2018), Atmospheric blocking as a traffic jam in the jet stream is now available on Science!

NH18 Science Paper Preview

Here is the press release from UChicago about the publication.

For interested researchers, the sample script to reproduce the results can be found in the directory nh2018_science of the my python package’s GitHub repo hn2016_falwa. You can download ERA-Interim reanalysis data with download_example.py to run the local wave activity and flux analysis in the jupyter notebook demo demo_script_for_nh2018.ipynb.

Have fun and feel free to email me (csyhuang at uchicago.edu) if you are interested in using the code and/or have questions about it.

Installing Stanford Core NLP package on Mac OS X

I am following instructions on the GitHub page of Stanford Core NLP under Build with Ant. To install ant, you can use homebrew:

$ brew install ant

In Step 5, you have to include the .jar files in the directory CoreNLP/lib and CoreNLP/liblocal in your CLASSPATH. To do this, first, I install coreutils:

brew install coreutils

such that I can use the utility realpath there. Then, I include the following in my ~/.bashrc:

for file in `find /Users/clare.huang/CoreNLP/lib/ -name "*.jar"`;
  do export CLASSPATH="$CLASSPATH:`realpath $file`";
done

for file in `find /Users/clare.huang/CoreNLP/liblocal/ -name "*.jar"`;
  do export CLASSPATH="$CLASSPATH:`realpath $file`";
done

(I guess there are better ways to combine the commands above. Let me know if there are.)

To run CoreNLP, I have to download the latest version of it, and place it in the directory CoreNLP/:

wget http://nlp.stanford.edu/software/stanford-corenlp-full-2018-01-31.zip

The latest version is available on their official website. Unzip it, and add all the .jar there to the $CLASSPATH.

Afterwards, you shall be able to run CoreNLP with the commands provided in the blogpost of Khalid Alnajjar (under Running Stanford CoreNLP Server). If you have no problem starting the server, you shall be able to see the interface on your browser at http://localhost:9000/:

java -mx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -annotators "tokenize,ssplit,pos,lemma,parse,sentiment" -port 9000 -timeout 30000

Yay. Next, I will try setting up the python interface.

Installing java on Mac

The information of this post was learnt from this StackOverflow post and also David Cai’s blog post on how to install multiple Java version on Mac OS High Sierra.

With brew cask installed on Mac (see homebrew-cask instructions), different versions of java can be installed via the command (I want to install java9 here, for example):

brew tap caskroom/versions
brew cask install java9

After installing, the symlink /usr/bin/java is still pointing to the old native Java. You can check where it points to with the command ls -la /usr/bin/java. It is probably pointing to the old native java path: /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java .

However, homebrew installed java into the directory /Library/Java/JavaVirtualMachines/jdkx.x.x_xxx.jdk/Contents/Home.

To easily switch between different java environments, you can use jEnv. The installing instructions can be found on jEnv’s official page.

my widget for counting (since Dec24, 2016)