Microsoft CNTK 2.0のチュートリアル試行 (SimpleMNIST.py)

CNTK 2.0にも、フィードフォワードのMNISTのサンプルが乗っている。これを試行して、中身を可能な限り理解してみよう。

まず、MNISTのデータセットを取得する必要があるのだが、これは以下のチュートリアルに則る。

github.com

cd Examples/Image/Classification/MLP/Python
cd ../../../DataSets/MNIST/
python install_mnist.py

次に、MNISTを実行してみよう。

cd ../../Classification/MLP/Python
python SimpleMNIST.py
Selected CPU as the process wide default device.
-------------------------------------------------------------------
Build info:

                Built time: May 31 2017 17:14:07
                Last modified date: Thu May 18 15:42:56 2017
                Build type: release
                Build target: CPU-only
                With 1bit-SGD: no
                With ASGD: yes
                Math lib: mkl
                Build Branch: HEAD
                Build SHA1: ade8bc05d30e61160da729aee078e22f8bd4fced
                Built by Source/CNTK/buildinfo.h$$0 on a4dbcb4fb82a
                Build Path: /home/philly/jenkins/workspace/CNTK-Build-Linux_2
                MPI distribution: Open MPI
                MPI version: 1.10.3
-------------------------------------------------------------------
Learning rate per sample: 1.0
Finished Epoch[1 of 10]: [Training] loss = 0.382948 * 60032, metric = 10.21% * 60032 3.174s (18913.7 samples/s);
Finished Epoch[2 of 10]: [Training] loss = 0.202842 * 59968, metric = 5.71% * 59968 2.002s (29954.0 samples/s);
Finished Epoch[3 of 10]: [Training] loss = 0.156752 * 60032, metric = 4.43% * 60032 2.065s (29071.2 samples/s);
Finished Epoch[4 of 10]: [Training] loss = 0.128525 * 59968, metric = 3.65% * 59968 2.157s (27801.6 samples/s);
Finished Epoch[5 of 10]: [Training] loss = 0.109642 * 60032, metric = 3.07% * 60032 2.014s (29807.3 samples/s);
Finished Epoch[6 of 10]: [Training] loss = 0.095389 * 59968, metric = 2.69% * 59968 2.171s (27622.3 samples/s);
Finished Epoch[7 of 10]: [Training] loss = 0.084236 * 60032, metric = 2.38% * 60032 2.415s (24858.0 samples/s);
Finished Epoch[8 of 10]: [Training] loss = 0.075827 * 59968, metric = 2.11% * 59968 2.411s (24872.7 samples/s);
Finished Epoch[9 of 10]: [Training] loss = 0.068614 * 60032, metric = 1.92% * 60032 2.438s (24623.5 samples/s);
Finished Epoch[10 of 10]: [Training] loss = 0.062529 * 59968, metric = 1.72% * 59968 2.296s (26118.5 samples/s);
Error: 0.025174

なんとなく動いた。これ何をやっているんだろう？

• Sequential (arrayOfFunctions)

これは関数をシーケンシャルに実行する関数だ。例えば以下のように記述すると、

FGH = Sequential (F:G:H)
y = FGH (x)

以下と等価である。

y = H(G(F(x)))

今回のMNISTでは、

    z = Sequential([For(range(num_hidden_layers), lambda i: Dense(hidden_layers_dim, activation=relu)),
                    Dense(num_output_classes)])(scaled_input)

という風に使われており、Forは繰り返し、DenseはFull-Connectedのレイヤを作る関数、つまりこれはnum_hidden_layersだけDenseを接続し、最後にnum_output_classesサイズのDenseを接続している。

次にトレーニング、最後にテストだ。

    # Instantiate the trainer object to drive the model training
    lr = learning_rate_schedule(1, UnitType.sample)
    trainer = Trainer(z, (ce, pe), adadelta(z.parameters, lr), progress_writers)

    training_session(
        trainer=trainer,
        mb_source = reader_train,
        mb_size = minibatch_size,
        model_inputs_to_streams = input_map,
        max_samples = num_samples_per_sweep * num_sweeps_to_train_with,
        progress_frequency=num_samples_per_sweep
    ).train()
...

    # Test data for trained model
    test_minibatch_size = 1024
    num_samples = 10000
    num_minibatches_to_test = num_samples / test_minibatch_size
    test_result = 0.0
    for i in range(0, int(num_minibatches_to_test)):
        mb = reader_test.next_minibatch(test_minibatch_size, input_map=input_map)
        eval_error = trainer.test_minibatch(mb)
        test_result = test_result + eval_error

    # Average of evaluation errors of all test minibatches
    return test_result / num_minibatches_to_test