nnx example

[manamir]

Hi, am very new with nnx and i was wondering if there's a simple example of training and testing an RNN with nnx library like it's the case with the nn library for a feed forward neural network?

[nicholas-leonard]

Hi @manamir you can use the partial example included in the doc : https://github.com/clementfarabet/lua---nnx#nnx.Recurrent .

There is also a complete working example in https://github.com/nicholas-leonard/dp/blob/master/examples/recurrentlanguagemodel.lua .

If you have any questions, don't hesitate to ask.

[manamir]

Hi @nicholas-leonard and thanks again for the very useful links.
Actually, i ran the recurrentlanguagemodel.lua script and it works fine. What i want to do now is to apply the script to my own data. As i understood, there's a way to represent data (word_freq, word_map, word_tree, etc.). So i was wondering if starting from a text data, there's a script to generate all the needed files (word_freq, word_map, word_tree, etc.) or should i create each file myself separately?

[nicholas-leonard]

@manamir What is your dataset? How many unique words in vocabulary?

The most important thing is to create the train_data.th7, valid_data.th7 and test_data.th7 files as they contain the actual words. Each of these is a tensor encapsulated by a SentenceSet.

[manamir]

hi @nicholas-leonard
My dataset is the PenTreeBank,
i guess i should create a script called PenTreeBank.lua in the same way as billionwords.lua and modify the recurrentlanguagemodel.lua script so that it takes into account PenTreeBand data right?
is train_data.th7 a binary file of a given train_data which is a raw text?
am a bit confused.

[manamir]

i have a raw training data and am still wondering how to convert it into .th7 file ?

[manamir]

ok as i understood i have to parse the raw data and assign to each sentence a corresponding id and than create train.th7 valid.th7 test.th7 and word_map.th7 corresponding to the sentence and words ids

[nicholas-leonard]

Yeah. I am actually looking to add this dataset to dp. Do you intend to submit a Pull Request for PenTreeBank?

[manamir]

yes i think so

[nicholas-leonard]

Awesome, if you need help finishing it. Just send a Pull Request with what
you have. I can help you with the rest.

Nicholas Léonard
450-210-1214

On Tue, Mar 17, 2015 at 10:39 AM, manamir notifications@github.com wrote:

yes i think so

Reply to this email directly or view it on GitHub
#29 (comment)
.

[manamir]

couldn't figure out how raw data is stored in .th7 file

[nicholas-leonard]

First, load data into a torch tensor of size N words x 2, i.e. a torch.tensor with 2 columns. First col is for storing start indices of sentences. Second col is for storing the sequence of words as shuffled sentences. Sentences are only seperated by the sentence_end delimiter, which is a unique integer of your choosing. So for example sentence A : {1,3,5,6} and sentence B {9,3,1,3}, using sentence delimiter 77, would be stored as :

torch.Tensor({{1,1},{1,3},{1,5},{1,6},{1,77},{6,9},{6,3},{6,1},{6,3},{6,77}})
  1   1
  1   3
  1   5
  1   6
  1  77
  6   9
  6   3
  6   1
  6   3
  6  77
[torch.DoubleTensor of dimension 10x2]

Then you save that tensor to disk using torch.save("path/to/file.th7", mytensor.

[manamir]

Thanks a lot, it works perfectly :)

[manamir]

Now am facing a memory size problem... the tensor size is ; (number of sentences x number of words x 2) this is huge

[manamir]

may be i should concatenate different tensor

[nicholas-leonard]

It should be a tensor of size 2 x total number of words. Not to be confused with 2 x number of unique words.