What is more important, a deep network weights or its activations? Obviously, we can derive the network’s activation from its weights. Yet, deep networks are non-linear embedding functions; we want this non-linear embedding only. On top of this embedding, we either slap a linear classifier in a classification network or…

While training a deep network on multiple tasks jointly is easy, training on multiple tasks sequentially is challenging. This challenge is addressed in various literature: Lifelong Learning, Incremental Learning, Continual Learning, and Never-Ending Learning. Within these forms, the common problem is catastrophic forgetting, i.e., the network forgets older tasks. To…

Deep learning stands on two pillars: GPUs and large datasets. Thus, deep networks suffer when trained from scratch on small datasets. This phenomenon is known as overfitting. This paper [1] proposes knowledge evolution to reduces both overfitting and the burden for data collection. In the paper, knowledge evolution is supported…

Every software engineer has used a debugger to debug his code. Yet, a neural network debugger… That’s news! This paper [1] proposes a debugger to debug and visualize attention in convolutional neural networks (CNNs).

Before describing the CNN debugger, I want to highlight few attributes of program debuggers (e.g., gdb)…

Thanks, Kubra, for sharing your thoughts. I never used Deep image prior. It is a cool idea, yet, its training cost is a barrier. As you mentioned, the total time spent on training is optimized; that's correct. Yet, this training time is incurred for every image, i.e., every image is a training sample, there is no inference.

Furthermore, I had a colleague who used deep prior for RGB-D images [1]. The training cost becomes more severe as the number of dimensions increases.

So, Yes, for 2D images the cost might be manageable, but I would proceed with caution for higher dimensions (e.g., videos). Thanks again for sharing your input :)

[1] Depth Completion Using a View-constrained Deep Prior

Ahmed Taha

I write reviews on computer vision papers. Writing tips are welcomed.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store