I've started a machine learning project using TemsorFlow. The first thing I've learned is to start small, implement all the functionality and only then move to a real task.

I picked up Keras, so the code for the network itself is simple. Complexity is around: dataset preparation, evaluation and model selection.

Show thread

Initially I worked on my laptop with an extremely small values, for example, images were resized to 128 by 128, which feels too small. However, I managed to get reasonable performance numbers.

Show thread

When I moved my code to a much more powerful machine with lots of memory, cpus and gpus, I got a GPU enabled TemsorFlow binary. That speed up the things, but still CPU was a bottleneck.

Show thread

Cpu during training or during preprocessing/data manipulation?

Training. I used PIL to resize images, so it's just CPU. Actually, Pillow-SIMD to get easy speedups.

Sign in to participate in the conversation
Scholar Social

Scholar Social is a microblogging platform for researchers, grad students, librarians, archivists, undergrads, academically inclined high schoolers, educators of all levels, journal editors, research assistants, professors, administrators—anyone involved in academia who is willing to engage with others respectfully.