Yr12 Journal 17

Comments

In Term 3 Week 4, I was making backend api for my project
This blog post contains lots of codes and the word count is definitely less than 1000.

Progress

Somehow I chose to make a website for the User Interface of my project. I think I accidently now combined web dev assignment with my data science assignment (pretty sure web dev people were asked to make a website as well). Maybe Mr. Griffin should consider give me extra marks due to I actually have done a project that covers to subjects?

So basically, I decided to make a complete website, with frontend and backend. I will be writing static HTML/JS/CSS files to make my frontend looks beautiful and ready to display my project as a user interface. Alternatively, I will use python (of course python because I wrote my Machine Learning Application in python) to make my backend service, it will provide several APIs, such as providing accuracy of the model, make prediction of a custom crypto coin price in a certain time, make prediction of custom crypto coin price in next 15 mins (to generate a chart in frontend).

I will use the following technology to make my website (please consider this as web dev assignment and give me extra credits in my data science project):

  • Frontend
    • Bootstrap -> beautiful interface
    • Chart.js -> illustrate my “prophecy”
    • JQuery.js -> ajax request to backend

as I will run frontend and backend on a same web service, no need for prevent CORS

  • Backend
    • Flask -> run the webservice

In this week I finished implementing my backend service as well as setting up flask environment. All my APIs were set up and can return JSON result.

Pretty much the most important bit of my backend service is according to this block of code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def get_price_min(coin, minute):
    coin = coin.upper()
    model_path = f"saved_model/{coin}_model"
    cur_b = d.get_avg_BTC()
    cur_p = d.get_avg_price_AUD(coin)
    if not u.exists(model_path):
        print(f"downloading {coin} data...")
        # get dataframe
        df = d.get_history_df(coin, 2022, 7)
        print(f"training model for {coin}...")
        # process and save model
        model = u.process_history_to_model(coin, df)
    else:
        print(f"loading {coin} model from cache...")
        model = u.load_model(model_path)
    # past data
    ck, bk, coin_k, btc_k = d.download_last_hr_data(coin)
    # error
    err = u.predict_price(model, coin, 1, ck, bk, coin_k, btc_k) * cur_b - cur_p
    err = err
    neg = err < 0
    print(f"err = {err}")
    if abs(err) < 0.01:
        err = abs(err) ** 0.5
        if neg:
            err = -err
        btc = u.predict_price(model, coin, minute, ck, bk, coin_k, btc_k) * (1 - err)
        aud = btc * cur_b
    else:
        btc = u.predict_price(model, coin, minute, ck, bk, coin_k, btc_k)
        aud = btc * cur_b + err * 10
    print(f"Prediction in next {minute} mins: 1 {coin} = {btc} BTC")
    print(f"Prediction in next {minute} mins: 1 {coin} = {aud} AUD")
    print(f"Prediction in next {minute} mins: {(aud / cur_p - 1) * 100}%")
    return (btc, aud)

This is the function that use my model and make prediction, where u.predict_price and other apis were what I did in the past weeks.

As long as the core function is defined, here are the implementations of my webservice APIs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
@app.route('/predict_<coin>/<int:minute>')
def predict(coin, minute):
    coin = coin.upper()
    try:
        btc, aud = get_price_min(coin, minute)
        return {
            'code': 200,
            'btc_price': float(btc),
            'aud_price': float(aud)
        }
    except:
        return {
            'code': 500
        }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
@app.route('/predict_<coin>_15')
def predict_15_min(coin):
    coin = coin.upper()
    try:
        btc = []
        aud = []
        for i in range(1, 15):
            b, a = get_price_min(coin, i)
            btc.append(b)
            aud.append(a)
        return {
            'code': 200,
            'btc_price': [float(b) for b in btc],
            'aud_price': [float(a) for a in aud],
        }
    except:
        return {
            'code': 500
        }
1
2
3
4
5
6
7
8
9
10
11
12
@app.route("/cur_btc")
def cur_btc():
    try:
        return {
            'result': 200,
            'price': d.get_avg_BTC()
        }
    except:
        return {
            'result': 500,
            'price': 0
        }
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
@app.route("/check_model_<coin>")
def check_model(coin):
    coin = coin.upper()
    model_path = f"saved_model/{coin}_model"
    print(f"downloading {coin} data...")
    # get dataframe
    df = d.get_history_df(coin, 2022, 7)
    # datasets
    o_train_dataset = u.get_train_dataset(df)
    o_test_dataset = u.get_test_dataset(df, o_train_dataset)
    o_train_labels = u.get_train_labels(o_train_dataset)
    o_test_labels = u.get_test_labels(o_test_dataset)
    # convert
    test_dataset = u.get_norm_df(o_test_dataset).to_numpy()
    test_labels = u.get_norm_df(o_test_labels).to_numpy()
    try:
        if not u.exists(model_path):
            print(f"training model for {coin}...")
            # process and save model
            model = u.process_history_to_model(coin, df)
        else:
            print(f"loading {coin} model from cache...")
            model = u.load_model(model_path)
        loss, mae, mse = model.evaluate(test_dataset, test_labels, verbose=2)
        return {
            'code': 200,
            'loss': loss,
            'mae': mae,
            'mse': mse,
        }
    except:
        return {
            'code': 500
        }
1
2
3
4
5
6
7
8
9
10
11
12
13
@app.route("/cur_<coin>")
def cur_cust_coin(coin):
    try:
        coin = coin.upper()
        return {
            'result': 200,
            'price': d.get_avg_price_AUD(coin)
        }
    except:
        return {
            'result': 500,
            'price': 0
        }

Those APIs will be used in the further frontend development in next week.

Challenges

The only reason is to build a M1 environment for tensorflow, because I use M1 laptop in school. Fortunately, apple have a repo on github that contains a shell script to compile arm structure tensroflow by one key. Here is the repo

Reflection

  • I planned how to develop my user interface
  • I successfully ran my project (MLP + Flask) on both my laptops (ARM and X64 structures)
  • I implemented my backend APIs

Timeline

I am on track, and still beyond my original plan. I will finish frontend development next week. After next week my project development will be finished, and I will start testing.