Working with new version of TF & Extracted Hand class to separate file.

[CT83]

I plan to move more stuff around in the coming days, and refactor it so that different styling (biases, line height, width, colors) can be managed easily. I am planning to turn this into something beginner ML Engineers like me can easily work on.

Great work on the Repo!🎉

[CT83]

I have also made few changes which allow the code to be compatible with current version of tensorflow. Please let me know what you think and if I should proceed with what I am doing and keep making PRs

[CT83]

I am just expanding on your work here, but this is what I am trying to perfect. I will send my changes in a PR if you are interested.

[mandystroech]

Can you add more writing styles/fonts too?

[CT83]

@mandystroech Head over to my Fork https://github.com/CT83/handwriting-synthesis . See https://github.com/CT83/handwriting-synthesis/blob/testing/ct83_demo.py Changing the styles/fonts is only a matter of changing the style, bias, line_height, stroke_widths you can get unlimited combinations using these. For anything else you would have to collect training data separately.

[TheMasterOfMagic]

@mandystroech Head over to my Fork https://github.com/CT83/handwriting-synthesis . See https://github.com/CT83/handwriting-synthesis/blob/testing/ct83_demo.py Changing the styles/fonts is only a matter of changing the style, bias, line_height, stroke_widths you can get unlimited combinations using these. For anything else you would have to collect training data separately.

Hi CT83, I'm new to this, and since the original author seems to have disappeared for quite a long time, I was going to your fork to post this. But I cannot find the Issues entry there so I post this here. Could you please solve my questions below?

1. Where do the style files come from? How to generate a new style?
2. I have trained my own model following the instructions mentioned in the readme.md file, and my question is how do I use it? Is it relevant to question 1?

Best regards, Newbie

[CT83]

Hey @TheMasterOfMagic,

1. I am probably in the same boat as you, I honestly don't know how the styles were generated, I will probably have to look into it myself, but I think maybe maybe you could try to view them using OpenCV's cv2.imshow() the styles seem to be numpy arrays and so are OpenCV images, I am not sure how that would work, but it's worth a try. You might have already read the paper this repo is based on Generating Sequences With Recurrent Neural Networks by Alex Graves does it shed any light on how this process works?

2. You could try looking more into the rnn.py, tf_base_model.py and the checkpoints directory, that is where the original model seems to be stored.

I was working on this project more from refactoring the code point of view I wished I was of more help. Also, I have just enabled issues on my Fork, you should see them now, but I would suggest keeping the issues here, as I more people will see those.

[TheMasterOfMagic]

Hey @CT83,

Thanks a lot !! And ~~with poor English,~~ here are my replies:

• About the codes, I have traversed all those files and searched the keywords style and save, but it seems that these two words never appear at the same time in the project. That's what brings me here today.
• I do know what .npy file is but I really didn't come up with cv2.imshow. Thanks for your in time reminding!
• I also have read the paper roughly, and I'm going to read it more seriously, and I will keep a mind on this side. Once I get something I'll post it here :)

One more time, thanks a lot for your in time reply!

[TheMasterOfMagic]

Hi @CT83 , I got some findings about what those style files are!! Last time, I np.load-ed those style files and still got nothing. It just seems that for each style, there is

• a "characters" numpy file, which is a sentence in English
• a "strokes" numpy file. which is a np.array with shape (n, 3). And for each raw, the first two values are always some small numbers around zero, and the third value is always a binary digit, mostly 0.

I used plt.imshow to show each strokes files but mess and dark is all I got. These days I've read the paper again and again, and I found something today! In the 3rd paragraph of subsection 4.1 Mixture Density Outputs on page 19, the paper says the first two values are "the pen offset from the previous input", and the third value defines if the current point ends a stroke(pen was lifted off the board).

I then calculated the cumulative sum of each strokes and I got images like below:

Each of them matches the sentence in their corresponding "characters" files. In other words, the sentences in the "character" files explain what the "strokes" files write.

But I still have no idea what the relationship between the NN and these style files is :(

[CT83]

Wow! You made some progress! Awesome, keep at it. I am glad to hear that the reading the paper helped. You could try joining some online groups like Slack Channels those usually are full of smart people, who can help. Google. Deep Learning Slack Groups