data files on disk

Cyrus Taylor (cct@po.cwru.edu)
Mon, 5 Feb 1996 10:59:04 -0500

There's been a lot of health discussion in the past week. I'd now like to throw in my 2 C worth.

Some of the items below involve questions to various people, so please read attentively, and
respond as appropriate.

Major issues I have noted include:

(1) The need to return the APEX space to them in a timely fashion. Jon, do I understand properly that
this needs to be done by the end of next week?

(2) The need [expressed by Mike Longo] to retain VAX processing capability. As I understand
it, we should expect that the VAX's will be terminated at the end of the run and that there is no chance of
maintaining them. To the best of my knowledge, maintaining a VAX processing option is not an option,
painful as that is to contemplate. If anyone can correct this perception, please do so.

(3) Can the disks on fnmini/fnmint be moved to FNMINE? If so, how soon? Soon enough to take
some of the load off of the APEX disks? FNMINE is on AFS, so that the data would be as accessible
as that on FNALU.

(4) Where will FNMINE move to at the end of the run? Bj is currently planning to be at Fermilab
the week of February 19 to sort out issues of this sort.

(5) Can we use other AFS resources? Bj has investigated the situation at SLAC, which is in
principle promising, but Jon seems unenthusiastic.

(5) Should we have a production run of Ken's dsts now? This is a moot point: a production run has
been made, using some 260 RU's on T3D at the Ohio Supercomputer Center. This was done as a
target of opportunity: due to an administrative snafu, the machine has been sitting completely
unused [except by us], permitting extremely rapid turn-around time. It was/is recognized that this
may need to be redone, but in the meantime, we are ahead of where we otherwise would be.

(6) Should we make dst's at all? Different people engage in different types of analysis. My pattern is
such that I make repeated passes through many files. The computation overhead on re-tracking, and
possibly even re-fitting, is terrible to contemplate, particularly given problems caused by other
users on FNALU. For the moment [i.e. until there is a compelling reason otherwise], I believe that
we must maintain a reasonbly complete set of .fsp and .dst files, accessible over AFS. It is OK
to compress the information in the dsts by writing unformatted rather than formatted, but we should
not drop information in either the .fsp or .dst files: it is all potentially essential at this point.

The bottom line is that as we move forward we may be able to come up with a better way of storing
the data than now, but I am very skeptical of this, given the diversity of analyses we contemplate.
I certainly oppose making such decisions under the time pressure imposed by the current disk crunch.

(7) What about GEANT? Similar comments apply.

(8) What about other trackers? As Tom's tracker is ready to enter production, we should make
sure that we have sufficient space for the dst's he makes as fast as he can make them. Given his
travel schedule, this seems unlikely to happen for at least the next six weeks. [Tom--correct me
if I am wrong]. While this is still reasonably short term, it is long-term compared to the time
in which we have to free up the APEX disk we are borrowing.

(9) What about Ken's proposal for freeing up space now? Who will be able to do the archiving to
tape? I actually Ken's proposal is fine, but I
would go farther. My own proposal would be to keep
(i) all of the recent collider runs and post-collider beam gas runs on disk, with dst's.
[that is, all runs 1087 and later]
(ii) a minimal set of reference/calibration runs from last spring/summer [with dst's]:
674,
677,
738,
739,
741,
747,
776,
779,
795,
797,
813,
814,
858,
868,
892,
921,
922,
927,
928,
929,
938,
939,
941,
942,
949,
950,
951,
986,
988,
989,
990,
991,
995,
995,
1000,
1001,
1002,
1003

-----------------------------------
Cyrus Taylor (216) 368-3710
Warren E. Rupp Associate Professor (216) 368-4671 (FAX)
Department of Physics cct@po.cwru.edu
Case Western Reserve University ctaylor@fnalv.fnal.gov
Cleveland, OH 44106-7079
USA