Master-Seminar - Deep Learning in Physics (IN2107, IN0014)

held by Prof. Dr. Nils Thuerey

Time, Place:

Mondays, 12:00-14:00 ONLINE

Kick-OFF: via Zoom
Time: Feb 9. 2021, time 09:30

Begin:

Monday, April 12., 2021 (online recording)

Details: Takes place online via BBB - Only online recordings until on-campus lectures possible again
Prerequisites: Introduction to Informatics I, Analysis, Linear Algebra, Game Physics and Introduction to Deep Learning recommended
Registration: Please register via the matching system

Content

Using deep learning methods for physical problems is a very quickly developing area of research. The research group of Prof. Thuerey has studied learning-based methods for Navier-Stokes problems and fluid flow applications in recent years, examples of which include learning latent-spaces for physical predictions, generative adversarial networks with temporal coherence, and the inference of Reynolds-averaged Navier-Stokes flows around airfoils. Beyond these physics-based deep learning works of the Thuerey group, this seminar will give an overview of recent developments in the field.

In this course, students will autonomously investigate recent research about machine learning techniques in the physical simulation area. Independent investigation for further reading, critical analysis, and evaluation of the topic are required.

Requirements

Participants are required to first read the assigned paper and start writing a report. This will help you prepare for your presentation.
Attendance
  • It is only allowed to miss a single time-slot. Missing a second one means failing the seminar. If you have to miss any, please let us know in advance.
Advisor
  • An advisor is assigned to each one with the paper.
  • Two weeks before the talk there will be a mandatory meeting with your advisor to review the report and discuss the structure of the presentation.
Report
  • A short report (4 pages max., excluding references in the ACM SIGGRAPH TOG format (acmtog) - you can download the precompiled latex template) should be prepared before the meeting with the advisor.
  • Guideline: You can begin with writing a summary of the work you present as a start point; but, it would be better if you focus more on your own research rather than just finishing with the summary of the paper. We, including you, are not interested in revisiting the work done before; it is more meaningful if you make an effort to put your own reasoning about the work, such as pros and cons, limitation, possible future work, your own ideas for the issues, etc.
Presentation (slides)
  • The participants have to present their topics in a talk (in English), which should last 30 minutes. Don't put too many technical details into the talk, make sure the audience gets the paper's main idea. Be prepared to answer questions regarding the technical details, you could prepare backup slides for that.
  • Afterwards, a short discussion session will follow.
  • Plagiarism is important; please do not simply copy the original authors' slides. You can certainly refer to them.
  • The semi-final slides (PDF) should be sent one week before the talk; otherwise, the talk will be canceled. We strongly encourage you to finalize the semi-final version as far as possible. We will take a look at the version and give feedback. You can revise your slides until your presentation.
  • Be ready in advance. We suggest testing the machines you are going to use before the lecture starts. You can bring your laptop or ask us one (also any converter you need for the projector) in advance. A laser pointer will be provided, so you can use if you want.
  • The final slides and report should be sent after the talk.

Virtual Seminar

BigBlueButton
  • For our seminar we use BigBlueButton: https://bigbluebutton.org/html5/
  • Please remain muted unless you have permission to speak. 
  • If you have a question or comment, please let us know in the chat, we will let you know as soon as you can speak, or post it directly in the chat.
Hardware Setup

A Laptop with built in microphone and speakers is terrible for everyone else in the virtual meeting!

  • Laptop fan makes noise (fan will most probably rev up after a couple of minutes, especially if you share your screen and if there are many users in the virtual meeting).
  • Typing and/or touchpad sounds will be transferred to communication partners.
  • Most likely there will be echo/feedback from the speakers.
  • Use the headset that came with your mobile phone. Even the cheapest headset will perform better than the built in microphone in a notebook.
  • (Cheap) bluetooth headsets are okay as well, but they perform not as good as wired ones!
  • Use an external microphone in combination with headphones. Some webcams have good quality microphones built in (e.g. Logitech).
  • Tablets like Apple iPads have quite solid built in microphones (and no fan). The same is possibly true for other tablets and even for smart phones. If you do not have a better alternative, they will most probably perform better than a Laptop.
Giving a Talk Online

Giving a talk in a virtual meeting scenario is quite different from giving a talk in a classroom scenario. You do not get any feedback from the audience during the talk (do they look happy/bored/satisfied?) and you cannot interact as easily in a non-verbal way with the audience as in the classroom, e.g. by pointing to things displayed on the projector screen. For this reason, mentally prepare yourself to talking to your screen for 20-30 minutes!

Normally, your slides should not contain too much written text. In the case of virtual meetings, some more information than usual is not wrong. Added info can help the audience to bridge gaps in case there was a glitch in the network connection, etc. Lastly, it is a good idea to open the content of the slides step by step to avoid that the audience can read faster than you talk.

Normally, when you give a talk, you are standing. This creates some tension in you, you will sound more energetic compared to talking while sitting, and, lastly, it helps you to concentrate. So it is a good idea to create a setup for giving the talk where you can stand in front of your screen.

You are supposed to give a talk. So do not write a script that you read to the audience. Reading out a pre-made script would be boring in a classroom scenario and it will be as boring in the virtual meeting!

Also important: switch off any type of messengers, notifications, etc. which can distract you. As you are pretty much isolated from the audience, it is a good idea to set a timer to keep track of elapsed time. Lastly, prepare for technical problems. Have some kind of second channel open to your advisor that can be used to contact you in case something unexpected happens.

Especially if you are using a quite “wide” (omnidirectional) microphone that sits on your desk, please avoid loud sounds from typing on your keyboard when you flip through slides while talking.

Preliminary Schedule

15. March 2021 Deregistration due    
28. March 2021 Deadline for sending an e-mail with 3 preferred topics    
29. March 2021 Introduction lecture    
31. March 2021 Notification of assigned paper    

10. May 2021

Yuanhao Zhong

Hamiltonian Neural Networks

Nilam T

10. May 2021

Ashwanth Ramesh

Machine learning accelerated computational fluid dynamics

You Xie

17. May 2021

Daniel Ziese

Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers

Liwei Chen

17. May 2021

Xingzhuo Yan

Learning data-driven discretizations for partial differential equations

You Xie

31. May 2021

Wing To Ku

Learning to control PDEs with differentiable physics

Liwei Chen

31. May 2021

Arian Bajrami

Discovering physical concepts with neural networks

Nilam T

07. June 2021

Tapish Narwal

Neural Ordinary Differential Equations

Liwei Chen

07. June 2021

Karan Shah

Deep learning and the Schrödinger equation

Nilam T

14. June 2021

Chenqi Zhou

Data-driven medium-range weather prediction with a Resnet pretrained on climate simulations: A new model for WeatherBench

You Xie

14. June 2021

Carlos Adrian Salas Cedillo

Data-driven nonlinear aeroelastic models of morphing wings for control

Liwei Chen

21. June 2021

Yujun Liu

tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow

You Xie

21. June 2021

Abdelrahman Amer

Solving high-dimensional partial differential equations using deep learning

Nilam T

28. June 2021

Eva Winker

Transfer learning for nonlinear dynamics and its application to fluid turbulence

Liwei Chen

28. June 2021

Christina Nuss-Brill

Deep learning methods for super-resolution reconstruction of turbulent flows

Nilam T

Papers

No. Paper  

1

Combining Differentiable PDE Solvers and Graph Neural Networks for Fluid Flow Prediction

https://arxiv.org/pdf/2007.04439.pdf

2

Machine learning accelerated computational fluid dynamics

https://arxiv.org/abs/2102.01010

3

Assessment of unsteady flow predictions using hybrid deep learning based reduced-order models

https://aip.scitation.org/doi/10.1063/5.0030137

4

Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers

https://arxiv.org/abs/2007.00016

5

Deep learning methods for super-resolution reconstruction of turbulent flows

https://aip.scitation.org/doi/full/10.1063/1.5140772

6

Learning data-driven discretizations for partial differential equations

https://www.pnas.org/content/pnas/116/31/15344.full.pdf

7

Learning to control PDEs with differentiable physics

https://arxiv.org/pdf/2001.07457.pdf

8

Discovering physical concepts with neural networks

https://arxiv.org/pdf/1807.10300.pdf

9

Neural Ordinary Differential Equations

https://arxiv.org/pdf/1806.07366.pdf

10

Hamiltonian Neural Networks

https://papers.nips.cc/paper/2019/file/26cd8ecadce0d4efd6cc8a8725cbd1f8-Paper.pdf

11

Physics Informed Deep Learning: Data-driven Solutions of Nonlinear Partial Differential Equations

https://arxiv.org/pdf/1711.10561.pdf

12

Deep learning and the Schrödinger equation

https://arxiv.org/pdf/1702.01361.pdf

13

Purely data-driven medium-range weather forecasting achieves comparable skill to physical models at similar resolution

https://arxiv.org/pdf/2008.08626.pdf

14

Model identification of reduced order fluid dynamics systems using deep learning

https://onlinelibrary.wiley.com/doi/epdf/10.1002/fld.4416?saml_referrer

15

Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows

https://arxiv.org/pdf/1810.08217

16

Data-driven nonlinear aeroelastic models of morphing wings for control

https://arxiv.org/pdf/2002.03139

17

tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow

https://arxiv.org/pdf/1801.09710

18

Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations

https://science.sciencemag.org/content/367/6481/1026.abstract

19

Lagrangian Fluid Simulation with Continuous Convolutions

https://openreview.net/pdf?id=B1lDoJSYDH

20

Solving high-dimensional partial differential equations using deep learning

https://www.pnas.org/content/115/34/8505

21

Transfer learning for nonlinear dynamics and its application to fluid turbulence

https://arxiv.org/pdf/2009.01407.pdf

22

SPNets: Differentiable Fluid Dynamics for Deep Neural Networks

https://arxiv.org/pdf/1806.06094.pdf

You can access the papers through TUM library's eAccess.