1nn tau emulator updates #73

djgagne · 2025-04-04T23:54:39Z

I am merging updates from Wayne Chuang and myself to the microphysics emulator code and to PUMAS itself. Updates in particular have been made to the quantile transform as well as to the order of variables in the input to the model.

Adding binary search to quantile transform

Fixed quantile transform

Index fix to tau emulator

commit 8e61d3f Merge: d2683f2 c9bff7e Author: Kate Thayer-Calder <[email protected]> Date: Mon Mar 10 16:38:26 2025 -0600 Merge pull request ESCOMP#71 from nusbaume/ccpp_prep Remove CAM dependencies and add CCPP layer for PUMAS v1 routines commit c9bff7e Author: Jesse Nusbaumer <[email protected]> Date: Fri Feb 28 10:54:10 2025 -0700 Add modifications requested during code review. commit a165913 Author: Jesse Nusbaumer <[email protected]> Date: Fri Feb 21 16:13:53 2025 -0700 Move variable initialization to inside GPU-ized loop. commit 685a20b Author: Jesse Nusbaumer <[email protected]> Date: Fri Feb 21 15:51:37 2025 -0700 Initialize local variables to zero (ESCOMP/PUMAS issue ESCOMP#72). commit 2b44b58 Merge: 7044b6b a8a10b6 Author: Jesse Nusbaumer <[email protected]> Date: Fri Feb 7 16:10:25 2025 -0700 Merge remote-tracking branch 'origin/ccpp_prep' into ccpp_prep commit 7044b6b Author: Jesse Nusbaumer <[email protected]> Date: Fri Feb 7 16:06:45 2025 -0700 Fix bad dimension in metadata file, and change subroutine dummy argument declarations to not specify dimension sizes. commit a8a10b6 Author: peverwhee <[email protected]> Date: Thu Jan 30 14:27:17 2025 -0700 fix stdnames for a couple of tendency variables commit fb7bbed Author: Jesse Nusbaumer <[email protected]> Date: Thu Jan 16 14:41:56 2025 -0700 Fix typo in comment. commit 7cde106 Author: Jesse Nusbaumer <[email protected]> Date: Thu Jan 16 08:47:49 2025 -0700 Add stub proc_rates DDT metadata file, fix comments, and update Tag Notes. commit fa085ce Author: Jesse Nusbaumer <[email protected]> Date: Tue Jan 14 16:18:50 2025 -0700 Fix errors found during a CAM run, and fix micro_pumas_ccpp.meta file using framework script. commit c5a15a3 Author: Jesse Nusbaumer <[email protected]> Date: Fri Jan 10 13:51:09 2025 -0700 Fix syntax errors found during compilation. commit fd86c7f Author: Jesse Nusbaumer <[email protected]> Date: Thu Jan 9 15:52:31 2025 -0700 Update dependency metadata listing to include wv_sat_methods.F90. commit 79cabd3 Author: Jesse Nusbaumer <[email protected]> Date: Wed Jan 8 16:06:45 2025 -0700 Finish CCPP interface for core PUMAS routines. commit 7a08cd5 Author: Jesse Nusbaumer <[email protected]> Date: Mon Dec 23 12:32:29 2024 -0700 Add remaining input and output arguments to CCPP run (tend) routine. commit 434778d Author: Jesse Nusbaumer <[email protected]> Date: Fri Dec 6 12:03:09 2024 -0700 Add input arguments to CCPP run (tend) routine. commit f820633 Author: Jesse Nusbaumer <[email protected]> Date: Tue Nov 19 15:29:46 2024 -0700 Add CCPP init phase, and perform additional clean-up. commit 42f2222 Author: Jesse Nusbaumer <[email protected]> Date: Wed Nov 13 11:16:57 2024 -0700 Remove unused 'micro_pumas_data.F90' file. commit 769f7a3 Author: Jesse Nusbaumer <[email protected]> Date: Thu Nov 7 15:46:25 2024 -0700 First attempt at removing/replacing non-portable CESM share code.

Katetc

Sorry I forgot to hit the green "finish review" button. Here's my issue with the iulog. I don't think this PR should come in until we are at least back to the emulated code working.

Katetc · 2025-06-26T22:14:03Z

micro_pumas_v1.F90

                                                  proc_rates%qrtend_TAU(1:mgncol,k), &
                                                  proc_rates%nctend_TAU(1:mgncol,k), &
-                                                  proc_rates%nrtend_TAU(1:mgncol,k))
+                                                  proc_rates%nrtend_TAU(1:mgncol,k), iulog)


So, we've removed iulog from the module and only pass it into the init functions of PUMAS as part of abstracting out of the CAM host model (and CCPP-izing). I traced this iulog and it is passed through to one write statement that seems unnecessary. Can we remove it all?

Katetc · 2025-06-26T22:16:10Z

module_neural_net.F90

    end subroutine quantile_inv_transform

-    subroutine neural_net_predict(input, neural_net_model, prediction)
+    subroutine neural_net_predict(input, neural_net_model, prediction, iulog)


Need to remove this iulog

Katetc · 2025-06-26T22:16:27Z

module_neural_net.F90

        integer :: input_size
        integer :: batch_index_size
        integer, allocatable :: batch_indices(:)
+        integer,  intent(in)  :: iulog


Remove this iulog

Katetc · 2025-06-26T22:17:26Z

tau_neural_net_quantile.F90

-    subroutine tau_emulated_cloud_rain_interactions(qc, nc, qr, nr, rho, lcldm, &
-            precip_frac, mgncol, q_small, qc_tend, qr_tend, nc_tend, nr_tend)
+    subroutine tau_emulated_cloud_rain_interactions(qc, nc, qr, nr, pgam, lamc, lamr, n0r, rho, lcldm, &
+            precip_frac, mgncol, q_small, qc_tend, qr_tend, nc_tend, nr_tend, iulog)


Need to remove this iulog

Katetc · 2025-06-26T22:17:39Z

tau_neural_net_quantile.F90

        real(r8), dimension(batch_size, num_inputs) :: nn_inputs, nn_quantile_inputs
        real(r8), dimension(batch_size, num_outputs) :: nn_quantile_outputs, nn_outputs
        real(r8), parameter :: dt = 1800.0_r8
+        integer,  intent(in) :: iulog


Remove iulog

Katetc · 2025-06-26T22:18:06Z

tau_neural_net_quantile.F90

                call quantile_transform(nn_inputs, input_scale_values, nn_quantile_inputs)
-                call neural_net_predict(nn_quantile_inputs, q_all, nn_quantile_outputs)
+
+                call neural_net_predict(nn_quantile_inputs, q_all, nn_quantile_outputs, iulog)


And here's the last iulog to remove, I think

nusbaume

@djgagne Thanks for trying to get this across the finish line! I agree with @Katetc about removing the iulog, and had a few questions/requests of my own, but hopefully none of them will be too difficult to resolve (and if you do have any questions or concerns just let me know). Thanks again!

nusbaume · 2025-07-25T03:09:43Z

module_neural_net.F90

+        real(kind = r8) :: mid_val
+        min_idx = 1
+        max_idx = size(x)
+        mid_val = 1


Do you need to initialize mid_val here outside of the while loop? It doesn't look like it is being initialized in binary_search_right below.

If it does need to be initialized, then I would recommend specifying the kind as well, i.e. mid_val = 1_r8.

tau_neural_net_quantile.F90

nusbaume · 2025-07-25T03:33:36Z

tau_neural_net_quantile.F90


    use module_neural_net, only : Dense, init_neural_net, load_quantile_scale_values
    use module_neural_net, only : quantile_transform, quantile_inv_transform, neural_net_predict
+    use tester, only : write_test_values


Is this still needed? It doesn't look like write_test_values is actually being called anywhere.

nusbaume · 2025-07-25T03:35:16Z

tester.F90

@@ -0,0 +1,25 @@
+module tester


If write_test_values is no longer used, then is this Fortran module file needed at all?

nusbaume · 2025-07-25T03:37:56Z

tester.F90

+    use shr_kind_mod,   only: r8=>shr_kind_r8
+    implicit none
+    integer, parameter, public :: i8 = selected_int_kind(18)


If you decide that this file would be beneficial to keep, then you'll need to make the following modification here in order to maintain PUMAS portability:

Suggested change

use shr_kind_mod, only: r8=>shr_kind_r8

implicit none

integer, parameter, public :: i8 = selected_int_kind(18)

use pumas_kinds, only: r8=>kind_r8

use pumas_kinds, only: i8=>kind_i8

implicit none

Wayne Chuang and others added 23 commits December 12, 2023 16:45

Tau emulator updated to output tendencies. Added qsmall for emulator

08482c4

bug fix for calling tau emulator

a9050fc

correcting order of emulator inputs

6df0b31

fix emulator output variables

31069d9

bugfix order of outputs qctend and qrtend

d763f63

Updated mass fixer

0edc0e2

Add test module to check quantile and nn outputs

1e1cb61

Remove excess code

e4fe824

Remove 1st timestep test that causes error

e0ec079

Remove additional 1st timestep test code

a617a8f

Merge branch 'master' of github.com:djgagne/PUMAS into 1nn_tau_emulator

679389b

Updated module neural net with binary search

84a73f9

Updated quantile transform

a13b820

Merge pull request ESCOMP#2 from djgagne/1nn_tau_emulator

907a49c

Adding binary search to quantile transform

Minor debugging for binary search

fe7e8e4

Debug val_idx

a00fdc8

Correcting val_index

fc34684

Fixed quantile transform

50d3c14

Merge pull request ESCOMP#3 from djgagne/1nn_tau_emulator

b8564a3

Fixed quantile transform

Fixed indexing error

7148e62

Merge pull request ESCOMP#4 from djgagne/1nn_tau_emulator

8b76bac

Index fix to tau emulator

Merge branch 'main_cam' of github.com:ESCOMP/PUMAS into 1nn_tau_emulator

479dd8d

Katetc mentioned this pull request Jun 13, 2025

Tau emulator updated to output tendencies. Added qsmall for emulator #65

Closed

Katetc requested review from Katetc and nusbaume June 24, 2025 23:02

Katetc mentioned this pull request Jun 24, 2025

New PUMAS tag coming with ML updates ESCOMP/CAM#1334

Open

Katetc requested changes Jul 1, 2025

View reviewed changes

Merge branch 'main_cam' of github.com:ESCOMP/PUMAS into 1nn_tau_emulator

f9dc513

nusbaume requested changes Jul 25, 2025

View reviewed changes

Removed commented code and iulog.

e1d3fa7

Katetc mentioned this pull request Sep 15, 2025

Integrate TAU Microphysics Emulator #66

Open

1nn tau emulator updates #73

Are you sure you want to change the base?

1nn tau emulator updates #73

Uh oh!

Conversation

djgagne commented Apr 4, 2025

Uh oh!

Katetc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nusbaume left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants