Skip to content

Commit 8cac71b

Browse files
feat: Address reviewer feedback (#7)
* Apply revisions to address reviewer feedback on 2024-09-30. - c.f. https://cds.cern.ch/record/2911566/comments?ln=en#C323903
1 parent e101d23 commit 8cac71b

File tree

3 files changed

+32
-18
lines changed

3 files changed

+32
-18
lines changed

figures/atlas-pipeline.png

396 KB
Loading

figures/coffea-casa.png

-314 KB
Loading

talk.md

+32-18
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ count: false
33

44
# Building a Columnar Analysis Demonstrator<br>for ATLAS PHYSLITE Open Data<br>using the Python Ecosystem
55
.large.blue[Matthew Feickert]<br>
6-
on behalf of ATLAS Analysis Model Group<br>
6+
on behalf of ATLAS Computing Activity<br>
77
.large[(University of Wisconsin-Madison)]
88
<br>
99
@@ -41,16 +41,16 @@ The infrastructure and methodology for these applications will be discussed in d
4141
<img src="figures/atlas-disk-projection.png" style="width:100%; box-shadow: 5px 5px 10px rgba(0, 0, 0, 0.5);">
4242
</a>
4343
</p>
44-
<!-- .caption[Projected evolution of disk under .blue[conservative] and .red[aggressive] R&D] -->
45-
.center.large[Moving forward wont be able to store everything on disk]
44+
.caption[([ATLAS Software and Computing HL-LHC Roadmap](https://cds.cern.ch/record/2802918), 2022)]
45+
.center.large[Moving forward won't be able to store everything on disk]
4646
]
4747
.kol-1-2[
4848
<p style="text-align:center;">
4949
<a href="https://indico.jlab.org/event/459/contributions/11586/">
5050
<img src="figures/physlite-model.png" style="width:90%; box-shadow: 5px 5px 10px rgba(0, 0, 0, 0.5);">
5151
</a>
5252
</p>
53-
.caption[[Jana Schaarschmidt](https://indico.jlab.org/event/459/contributions/11586/), CHEP 2023]
53+
.caption[(Jana Schaarschmidt, [CHEP 2023](https://indico.jlab.org/event/459/contributions/11586/))]
5454

5555
.center.bold[PHYSLITE]
5656
* Common file format for Run 4 Analysis Model
@@ -172,11 +172,11 @@ Providing the elements of an analysis pipeline
172172

173173
.kol-1-3[
174174
.large[
175-
* [University of Chicago Analysis Facility](https://af.uchicago.edu/) .bold[provides testing bed] with Coffea-casa
175+
* [University of Chicago Analysis Facility](https://af.uchicago.edu/) .bold[provides testing bed] with [Coffea-casa](https://iris-hep.org/projects/coffea-casa.html)
176176
* Provides support for:
177-
- .bold[JupyterLab] as a common interface
178-
- Highly efficient data delivery with .bold[XCache]
179-
- Conversion to columnar formats with .bold[ServiceX]
177+
- [.bold[JupyterLab]](https://jupyterlab.readthedocs.io/) as a common interface
178+
- Highly efficient data delivery with [.bold[XCache]](https://slateci.io/XCache/)
179+
- Conversion to columnar formats with [.bold[ServiceX]](https://iris-hep.org/projects/servicex.html)
180180
* Excellent integration exercise between analysis and operations
181181
]
182182
]
@@ -187,16 +187,29 @@ Providing the elements of an analysis pipeline
187187
</a>
188188
</p>
189189

190-
.caption[Platform for interactive analysis]
190+
.center.large[Scalable platform for interactive (or noninteractive) analysis]
191191
]
192192

193193
---
194194
# Structure of an ATLAS AGC
195195

196+
.kol-1-5[
197+
<br>
198+
<br>
199+
End user analysis ideally uses .bold[smaller and calibrated PHYSLITE]
200+
<br>
201+
<br>
202+
<br>
203+
<br>
204+
<br>
205+
.bold[Can still use PHYS] (same data format) through will need to perform .bold[additional steps] (calibration) with funcADL
206+
]
207+
.kol-4-5[
196208
<p style="text-align:center;">
197-
<img src="figures/atlas-pipeline.png"; width=80%>
209+
<img src="figures/atlas-pipeline.png"; width=100%>
198210
.center.large[Components of an ATLAS AGC demonstrator pipeline]
199211
</p>
212+
]
200213

201214
---
202215
# ATLAS Open Data
@@ -217,12 +230,11 @@ Providing the elements of an analysis pipeline
217230
<!-- box-shadow: 5px 5px 15px rgba(0, 0, 0, 0.5); adds a shadow that is 5px to the right and 5px down from the image, with a blur radius of 15px and a semi-transparent black color (rgba(0, 0, 0, 0.5)). -->
218231
<p style="text-align:center;">
219232
<a href="https://atlas.cern/Updates/News/Open-Data-Research">
220-
<img src="figures/atlas-open-data-announcement.png" style="width:100%; box-shadow: 5px 5px 10px rgba(0, 0, 0, 0.5);">
233+
<img src="figures/atlas-open-data-announcement.png" style="width:90%; box-shadow: 5px 5px 10px rgba(0, 0, 0, 0.5);">
221234
</a>
222235
</p>
223-
236+
.center[([ATLAS News, 2024-07-01](https://atlas.cern/Updates/News/Open-Data-Research))]
224237
<!-- .caption[13 TeV open data release from ATLAS] -->
225-
226238
<p style="text-align:center;">
227239
<a href="https://atlas.cern/Updates/News/Open-Data-Research">
228240
<img src="figures/open-access-principles.png" style="width:100%; box-shadow: 5px 5px 10px rgba(0, 0, 0, 0.5);">
@@ -260,7 +272,7 @@ Providing the elements of an analysis pipeline
260272
.large[
261273
* As columnar analysis .bold[processes events in batches] also need CP tools and algorithms to process in batches
262274
* Current CP tools operate on xAOD event data model (EDM) for calculation and write systematics to disk for future access (I/O heavy)
263-
* Challenge: Columnar on-the-fly computation be faster than disk
275+
* Challenge: Can columnar on-the-fly computation be faster than disk?
264276
* Refactoring to columnar studies in ATLAS AMG show .bold[improvements in performance and flexibility]
265277
]
266278
]
@@ -312,7 +324,7 @@ Providing the elements of an analysis pipeline
312324

313325
.kol-1-2[
314326
.large[
315-
Using zero-copy Python bindings to Egamma CP tool prototype
327+
Using zero-copy Python bindings to Egamma CP tool [prototype](https://gitlab.cern.ch/gstark/pycolumnarprototype/-/blob/58024df21af7d4465109fb668900567a3d0048c1/Zee_demo.ipynb)
316328
]
317329

318330
```python
@@ -344,9 +356,9 @@ Ongoing integration work into ATLAS Athena
344356
.kol-1-2[
345357
.large[
346358
* Tooling ecosystem is proving approachable and performant
347-
* Enabling university students to implement versions of the AGC by themselves with mentorship in a Jupyter notebook
348-
* ATLAS IRIS-HEP Fellow Denys Klekots's [AGC project using ATLAS open data](https://indico.cern.ch/event/1455396/contributions/6126406/) ([implementation on GitHub](https://github.com/iris-hep/agc-physlite))
349-
* Simplified version of [IRIS-HEP AGC top reconstruction challenge](https://agc.readthedocs.io/) using 2025+2016 Run 2 Monte Carlo from the 2024 ATLAS open data release
359+
* Enabling mentored university students to implement versions of the AGC by themselves in a Jupyter notebook
360+
* ATLAS IRIS-HEP Fellow Denys Klekots's [AGC project using .bold[ATLAS open data]](https://indico.cern.ch/event/1455396/contributions/6126406/) ([implementation on GitHub](https://github.com/iris-hep/agc-physlite))
361+
* Simplified version of [IRIS-HEP AGC top reconstruction challenge](https://agc.readthedocs.io/) using 2025+2016 Run 2 Monte Carlo from the 2024 .bold[ATLAS open data] release
350362
]
351363
]
352364
.kol-1-2[
@@ -364,6 +376,7 @@ Ongoing integration work into ATLAS Athena
364376
<img src="figures/denys_agc_ht.png" style="width:49%; box-shadow: 5px 5px 10px rgba(0, 0, 0, 0.5);">
365377
</a>
366378
</p>
379+
.center.large.bold[ATLAS open data]
367380
]
368381

369382
---
@@ -386,6 +399,7 @@ class: end-slide, center
386399
---
387400
# References
388401

402+
* [ATLAS Software and Computing HL-LHC Roadmap](https://cds.cern.ch/record/2802918), ATLAS Collaboration, 2022
389403
* [Using Legacy ATLAS C++ Calibration Tools in Modern Columnar Analysis Environments](https://indico.cern.ch/event/1330797/contributions/5796636/), Matthias Vigl, [ACAT 2024](https://indico.cern.ch/event/1330797/)
390404
* [How the Scientific Python ecosystem helps answering fundamental questions of the Universe](https://cfp.scipy.org/2024/talk/KCXVVR/), Vangelis Kourlitis, Matthew Feickert, and Gordon Watts, [SciPy 2024](https://www.scipy2024.scipy.org/)
391405
* [The Columnar Analysis Grand Challenge Demonstrator](https://indico.cern.ch/event/1268248/contributions/5326293/), Gordon Watts, [ATLAS S&C Plenary Afternoon: Demonstrators](https://indico.cern.ch/event/1268248/), 2023-10-04 [ATLAS Internal]

0 commit comments

Comments
 (0)