Aleksandar Jelenak presented benchmark results comparing an original and cloud optimized versions of the same file when either in local file system or AWS S3.This session was recorded on November 19, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle ...
The HDF Group’s John Readey hosted Call the Doctor today, Tuesday, November 12. In this session, John discussed field selection and data streaming with h5py and h5pyd.This session was recorded on November 12, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help user...
The HDF Group's director of engineering continued the discussion on HDF5 2.0. You can check out the planned work at the HDF5 2.0 Planning wiki. Projects for HDF5 2.0 include: semantic versioning, complex number support, and dropping the autotools and a few more rarely used things.This session was recorded on October 29, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, ...
The HDF Group's executive director Gerd Heber made himself available for community questions and highlighted another great community project, Versioned HDF5.Check out Gerd's notes here: https://forum.hdfgroup.org/t/hdf-clinic-10-29-2024/12753 This session was recorded on October 29, 2024. You can also watch this session online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week...
Scot Breitenfeld of The HDF Group made himself available for user questions and reviewed a recent comparison of hyperslab selection versus point selection in an HPC context.This session was recorded on October 22, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help...
In this session of Call the Doctor, The HDF Group's Director of Engineer Dana Robinson explained why the next (scheduled for April 2025) release of HDF5 will be HDF5 2.0 and what changes you can expect to see in this next major release.This session was recorded on October 15, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF ...
The HDF Group's John Readey hosted Call the Doctor. He followed up on his topic last month, which was fancy indexing. You can watch that video to catch up. On the forum, a community user did some benchmarking for the new feature. Make sure you check out those results. Next, John talked about recent updates to h5pyd. He discussed hsload, which provides improvements for hdf5 files with many small objects. and h5acl, which allows you to set access control.This session was recorded on October 8, ...
The HDF Group's Dana Robinson talked about the next release of HDF5 and why we're debating whether it will be 1.16 or 1.14.6. He also briefly talked about the upcoming HEP - HDF Enhancement Process and how that will help provide transparency and access to the project.This session was recorded on October 1, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, g...
Scot Breitenfeld hosted Call the Doctor on September 24. Scot's our HPC and Fortran expert and shared some HPC news including some testing on Aurora. There was a question and ensuing discussion on determining chunking parameters for using compression on parallel output.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions inten...
The HDF Group's Aleksandar Jelenak hosted Call the Doctor on Tuesday, 9/17 and talked about a Python script he wrote for additional (compared to h5stat) dataset statistics for a given HDF5 file. Aleksandar uses this script to inspect the changes after cloud optimizing a file.This session was recorded on September 17, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for...
The HDF Group's John Readey used his Call the Doctor session to talk about SWMR (single writer multiple reader) and FancySelections for h5pyd with HSDS. Example notebooks referenced by John during this session: https://github.com/HDFGroup/h5pyd/blob/master/examples/notebooks/swmr_example.ipynbhttps://github.com/HDFGroup/h5pyd/blob/master/examples/notebooks/fancy_selection.ipynbThis session was recorded on September 10, 2024. You can also watch this episode online. ...
The HDF Group's Director of Engineering Dana Robinson provided some general updates. Dana talked about upcoming releases of HDF5, HDF4, and HDFView. Communications Coordinator Lori Cooper announced the availability of the 2024 HDF5 User Group Meeting video recordings and talked briefly about The HDF Group's new support site (still a work in progress) at https://support.hdfgroup.org.Links mentioned during this session: HUG24 PlaylistHUG24 Videos and slide decks posted on the agendaForum p...
In addition to answering community questions, we will examine the latest PureHDF release(s). PureHDF v1.0.0 and v2.0.0 were released on June 24 (2024), and there have been subsequent minor releases (see Releases · Apollo3zehn/PureHDF · GitHub). We will examine what has changed since v1.0.0-beta.2 and use this opportunity to update the HDF5 tutorial (GitHub - HDFGroup/hdf5-tutorial: A tutorial for new and intermediate HDF5 users (of all ages)).This session was recorded on July 30, 2024. You ca...
The HDF Group's Scot Breitenfeld ended up using this session to discuss topics including subfiling, support of multidimensional arrays by various languages, the future of Fortran and more with community members and The HDF Group co-workers. In this session, we mentioned Aleksandar Jelenak's Call the Doctor session last week where he discussed Hidefix, a Rust package for multithreaded read-only access to local HDF5/netCDF4 files. The link was provided in chat and is being provided here f...
The HDF Group’s Aleksandar Jelenak will talk about a Hidefix, a Rust package for multithreaded read-only access to local HDF5/netCDF4 files. https://github.com/gauteh/hidefix It can also be imported as a Python package and used as an xarray backend engine. Aleksandar will also be available for user questions–he has a background as a scientist working with geospatial and geoscience data, and now works at The HDF Group to make HDF5 and related tools more effective for other scientists, speciali...
The HDF Group's senior informatics architect, Aleksandar Jelenak talked about some hdf5lib changes in the pipeline that will bring improvements for Cloud Optimized HDF5 files.This session was recorded on July 16, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help ...
The HDF Group's John Readey used his Call the Doctor session to talk about performance enhancements using the h5pyd MultiManager.For a use case where MultiManager came in clutch, check out this recent forum thread.This session was recorded on July 9, 2024. Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to h...
The HDF Group's Engineering Director Dana Robinson hosted the first Call the Doctor of the month and used his time to talk about upcoming releases and some activities and thoughts we’ve been working through on versioning. Then he talked about plans for a community proposal or HDF5 improvement system similar to the Python Improvement Process. This session was recorded on June 25, 2024. You can watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Gro...
In this episode of Call the Doctor, The HDF Group's Gerd Heber discusses the evolution of HDF documentation, highlighting challenges and past approaches and describes our shift towards treating documentation like code, citing the adoption of tools like Doxygen and plans to streamline documentation under a unified support platform. He previews upcoming updates to enhance readability and functionality, aiming for a rollout by the the HDF Users Group meeting in August. The session underscores th...
Aleksandar Jelenak of The HDF Group hosted Call the Doctor and shared his experiences with the s5cmd command-line tool that makes working with S3 object stores much easier and faster. He likes it and thinks you will too! This session was recorded on June 18, 2024. You can also watch this episode online.Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. ...