Help needed with tng

Discussion:

Help needed with tng_io library

Semen Esilevsky

2014-08-07 10:16:07 UTC

Dear All,
I'm currently trying to add support for tng files to my code and I realised that I can't understand the logic of the library at all. I'm definetely doing something wrong, but can't figure out a mistake.
The tng code is got from the latest git. Test trajectory is generated by Gromacs 5.0.

I can open the tng file and can extract, for example, position exponential correctly, but after that...

Problem #1: getting number of frames.

int64_t n1,n2;
tng_num_frames_get(trj, &n1);
tng_num_frame_sets_get(trj, &n2);

This gets
n1 = 10761
n2 = 1086

The correct number of frames is 1086.

What on Earth is returned from tng_num_frames_get()?

Problem #2: Reading past the end of file.

stat = tng_util_particle_data_next_frame_read(trj, TNG_TRAJ_POSITIONS, &values,
                                                  &datatype, &frame, &physical_time);

This function should return false when no more frames are available, right? It doesn't do this and reads forever returning garbage in values when called in a loop. stat is always true. How should I use it correctly to finish reading at the end of file?
Also concerning the strange thing with the number of frames and frame sets - what it reads actually each time?

Problem #3: tng_util_pos_read_range() behaves insane

int n=0;
bool stat = true;
while(stat){
    stat = tng_util_pos_read_range(trj,n,n, &ptr, &len);
    n++;
}

This always stops at n=10 (why?!), while the number of frames is 1086! What I'm doing wrong here?

I've also tried VMD plugin from pre-release VMD 1.9.2. It is said that this plugin is contributed by Gromacs developers, so it should work in principle. It doesn't work because of the bug #2 - it reads forever beyond the end of file.

The test tng file is here: https://drive.google.com/file/d/0Bx_ng_72VH8BQlFsTGJOVC1oMjQ/edit?usp=sharing

Any help is appreciated!

Sincerely,
Semen

--
Gromacs Developers mailing list

* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before posting!

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers or send a mail to gmx-developers-***@gromacs.org.

Magnus Lundborg

2014-08-07 10:37:26 UTC

Permalink

Hi,

I'll look into this when I'm back from vacation next week.

Cheers,

Magnus

Post by Semen Esilevsky
Dear All,
I'm currently trying to add support for tng files to my code and I
realised that I can't understand the logic of the library at all. I'm
definetely doing something wrong, but can't figure out a mistake.
The tng code is got from the latest git. Test trajectory is generated by Gromacs 5.0.
I can open the tng file and can extract, for example, position exponential
correctly, but after that...
Problem #1: getting number of frames.
int64_t n1,n2;
tng_num_frames_get(trj, &n1);
tng_num_frame_sets_get(trj, &n2);
This gets
n1 = 10761
n2 = 1086
The correct number of frames is 1086.
What on Earth is returned from tng_num_frames_get()?
Problem #2: Reading past the end of file.
stat = tng_util_particle_data_next_frame_read(trj, TNG_TRAJ_POSITIONS, &values,
&datatype, &frame,
&physical_time);
This function should return false when no more frames are available,
right? It doesn't do this and reads forever returning garbage in values
when called in a loop. stat is always true. How should I use it correctly
to finish reading at the end of file?
Also concerning the strange thing with the number of frames and frame sets
- what it reads actually each time?
Problem #3: tng_util_pos_read_range() behaves insane
int n=0;
bool stat = true;
while(stat){
stat = tng_util_pos_read_range(trj,n,n, &ptr, &len);
n++;
}
This always stops at n=10 (why?!), while the number of frames is 1086!
What I'm doing wrong here?
I've also tried VMD plugin from pre-release VMD 1.9.2. It is said that
this plugin is contributed by Gromacs developers, so it should work in
principle. It doesn't work because of the bug #2 - it reads forever beyond
the end of file.
https://drive.google.com/file/d/0Bx_ng_72VH8BQlFsTGJOVC1oMjQ/edit?usp=sharing
Any help is appreciated!
Sincerely,
Semen
--
Gromacs Developers mailing list
* Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
posting!
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers

Magnus Lundborg

2014-08-08 09:07:58 UTC

Permalink

Hi Semen,

I found some extra time, so I will try to clarify some things below.

One thing that is slightly difficult with TNG is that different kinds of
data can be written at different intervals, e.g. it can have positions
written every 100th frame, forces every 200th frame, velocities every
250th frame etc. To make this a bit more graspable in GROMACS every md
step is considered a frame when writing the TNG file (it would be
possible to let the n steps between frames be the greatest common
denominator of the written data types as well). The default when writing
TNG from a GROMACS simulation is to keep 100 frames of the most
frequently written data in each frame set. When generating a TNG from
trjconv (like the file you provided) it is not as easy to predict what
data to expect and the output frequency of it. Therefore a frame set is
created every time data is written - this will probably be changed in
the future to facilitate more efficient compression from trjconv too.

Post by Semen Esilevsky
Problem #2: Reading past the end of file.
stat = tng_util_particle_data_next_frame_read(trj, TNG_TRAJ_POSITIONS, &values,
&datatype, &frame, &physical_time);
This function should return false when no more frames are available, right? It doesn't do this and reads forever returning garbage in values when called in a loop. stat is always true. How should I use it correctly to finish reading at the end of file?
Also concerning the strange thing with the number of frames and frame sets - what it reads actually each time?
Problem #3: tng_util_pos_read_range() behaves insane
int n=0;
bool stat = true;
while(stat){
stat = tng_util_pos_read_range(trj,n,n, &ptr, &len);
n++;
}
This always stops at n=10 (why?!), while the number of frames is 1086! What I'm doing wrong here?

I will have a look at this, and will try to give more details next week
when I've got more time.

Post by Semen Esilevsky
I've also tried VMD plugin from pre-release VMD 1.9.2. It is said that this plugin is contributed by Gromacs developers, so it should work in principle. It doesn't work because of the bug #2 - it reads forever beyond the end of file.

This bug was discovered a few days after the release and is fixed in the
TNG library in the TNG repository. The fix is included in the
release-5-0 branch of GROMACS, but I don't think there has been any bug
fix release. I also think it will be fixed in the next VMD pre-release.

Post by Semen Esilevsky
The test tng file is here: https://drive.google.com/file/d/0Bx_ng_72VH8BQlFsTGJOVC1oMjQ/edit?usp=sharing
Any help is appreciated!

I hope the answers help you at least a bit. Please tell me if anything
is unclear and I will try to explain it better.

Kind Regards,

Magnus

Post by Semen Esilevsky
Sincerely,
Semen

Semen Esilevsky

2014-08-08 09:43:44 UTC

Permalink

Dear Magnus,
Thank you very much for explanations!

So, is there any boolet-proof way of finding out how many position sets are in the trajectory?

Another question is which function should I preferebly use to get the positions - tng_util_particle_data_next_frame_read() or tng_util_pos_read_range()? The later gives me an array of floats, which I need, but I suspect that it is much slower because of seeking for frame. Is it so?

I'm looking forward to see my stupid mistake, which leads to problems #2 and #3.

Regards,
Semen

Post by Magnus Lundborg
Hi Semen,
I found some extra time, so I will try to clarify some things below.

I will have a look at this, and will try to give more details next week
when I've got more time.

Post by Semen Esilevsky
The test tng file is here: https://drive.google.com/file/d/0Bx_ng_72VH8BQlFsTGJOVC1oMjQ/edit?usp=sharing
Any help is appreciated!

I hope the answers help you at least a bit. Please tell me if anything
is unclear and I will try to explain it better.
Kind Regards,
Magnus

Post by Semen Esilevsky
Sincerely,
Semen

Magnus Lundborg

2014-08-12 08:16:31 UTC

Permalink

Hi Semen,

There are currently no bullet-proof way of counting the number of frames
containing a specific type of data, but I'm working on adding that now.
To get the positions I would say that the easiest way is to call
tng_util_particle_data_next_frame_read() in a loop. That way you will
read until you find the next frame containing data of the requested type
(e.g. positions). When there is no more data you should get a TNG_FAILURE.

The bug #2 is the same as I mentioned as being fixed below, i.e. the
same bug that made VMD read forever. That should be fixed in the TNG
repository, otherwise I'll have to check what's wrong. I haven't checked
what's going on with problem #3 yet. I'll see when I have time to look
into that. Hopefully you can at least get further now. Otherwise just
ask me for more assistance and I can hopefully point you in the right
direction.

Cheers,

Magnus

Post by Semen Esilevsky
Dear Magnus,
Thank you very much for explanations!
So, is there any boolet-proof way of finding out how many position sets are in the trajectory?
Another question is which function should I preferebly use to get the positions - tng_util_particle_data_next_frame_read() or tng_util_pos_read_range()? The later gives me an array of floats, which I need, but I suspect that it is much slower because of seeking for frame. Is it so?
I'm looking forward to see my stupid mistake, which leads to problems #2 and #3.
Regards,
Semen

Post by Magnus Lundborg
Hi Semen,
I found some extra time, so I will try to clarify some things below.

I will have a look at this, and will try to give more details next week
when I've got more time.

Post by Semen Esilevsky
The test tng file is here: https://drive.google.com/file/d/0Bx_ng_72VH8BQlFsTGJOVC1oMjQ/edit?usp=sharing
Any help is appreciated!

I hope the answers help you at least a bit. Please tell me if anything
is unclear and I will try to explain it better.
Kind Regards,
Magnus

Post by Semen Esilevsky
Sincerely,
Semen