-
Notifications
You must be signed in to change notification settings - Fork 23
/
compression.doc
302 lines (249 loc) · 12 KB
/
compression.doc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
Notes on Bob Kerr's Data Compression Algorithm
as applied to two DNS data sets computed by
LANL's Turbulence Working Group (TWG).
For more information: contact Mark Taylor at [email protected] or
1. Description of Data Sets
These data sets are briefly described in S. Kurien and M.A. Taylor,
"Direct Numerical Simulation of Turbulence - Data Generation and
Statistical Analysis", Los Alamos Science, 29 (2005) 142--151. The
data contains snapshots of the velocity field, from DNS simulations in
a triply periodic box. The box has sides of length 1. Note that many
Fourier DNS codes assume a box of length 2 pi. If you are using such
a code, you must rescale this data accordingly.
For each snapshot, we include the KE and enstrophy. This data (and
other statistics) was also computed at every timestep during the
simulations, and is available by email from Mark Taylor. Recomputing
the KE and enstrophy is a good way to check if you have correctly read
in and uncompressed the data. To verify the integrety of the
compressed data files, we have included the md5 checksum of every
file in "md5sum.out". This was computed on QSC (the machine used
to compresses the data). Then the data was copied to the USB disks attached
to a Linux workstation and the md5 sums were verified again on this
system.
1A. 2048^3 data.
This data set is modeled after the upgraded Corrsin Wind Tunnel
experiment, (Kang, Chester and Meneveau, "Decaying turbulence in an
active-grid-generated flow and comparisons with large-eddy
simulation", J. Fluid Mech. 480, 2003). We start with an initial flow
field with random uncorrelated phases but a prescribed energy
spectrum. The flow is then evolved for a short time until the phases
become correlated enough to give a reasonable mean-derivative
skewness. The energy spectrum is then reinitialized back to the
original spectrum while retaining the correlated phases. This
reinitialization was done at t=0.4020, and thus we only include
snapshots from t>=0.4020. The simulation was then run out to t=3.79.
(The t=0.4020 snapshot the snapshot just before the renormalization.
It is provided as a reference and should not be considered as part of
the simulation)
Duration: t=.4020 to 3.7900
R_lambda: 181 at t=.4020, peaks at 208 and then decays to 152
Eddy turnover time: .89 at t=.4020, increasing to 6.5
This data is stored on disk1 and disk2 in the "decay2048" directories.
Filenames are of the form:
decay2048????.????.uc
decay2048????.????.vc
decay2048????.????.wc
where ????.???? is the simulation time.
time KE Enstrophy
0.4020 2.210360120232e-02 7.818009066737e+03
0.4026 3.805026964614e-02 2.288941791091e+04
0.4188 3.690181533067e-02 1.745150203958e+04
0.4328 3.606695173845e-02 1.621522156075e+04
0.4603 3.452270959126e-02 1.553188584727e+04
0.4894 3.294043690046e-02 1.514102172733e+04
0.5551 2.955198030044e-02 1.387420079469e+04
0.6034 2.728597472575e-02 1.261006134060e+04
0.6491 2.535662077989e-02 1.134070659993e+04
0.7019 2.338371035070e-02 9.930643655649e+03
0.7536 2.169650906123e-02 8.694657451604e+03
0.8149 1.996812635833e-02 7.449945780811e+03
0.8545 1.898535058904e-02 6.764576032006e+03
0.9040 1.788378475897e-02 6.021728419087e+03
0.9512 1.694561686200e-02 5.418153089714e+03
1.0017 1.604360384774e-02 4.866191416556e+03
1.0511 1.524901639655e-02 4.407402717750e+03
1.1037 1.448278502481e-02 3.988621827836e+03
1.1598 1.374714454786e-02 3.607994618901e+03
1.1959 1.330904724752e-02 3.392609822420e+03
1.2500 1.270122071920e-02 3.106035639451e+03
1.3081 1.210466399449e-02 2.837619649946e+03
1.3456 1.174611063741e-02 2.683091178888e+03
1.4034 1.123225224567e-02 2.469725981745e+03
1.4434 1.090029173299e-02 2.337361211547e+03
1.5038 1.043222914570e-02 2.157720782570e+03
1.5439 1.014065629304e-02 2.049368075628e+03
1.6124 9.676167983702e-03 1.882656844023e+03
1.6586 9.384490984754e-03 1.781085565116e+03
1.7075 9.092082630943e-03 1.682404965067e+03
1.7569 8.813745527858e-03 1.590554590799e+03
1.8484 8.336451113597e-03 1.437896164558e+03
1.8988 8.092885982495e-03 1.362803214517e+03
1.9493 7.861832544977e-03 1.293418865501e+03
1.9955 7.660721577368e-03 1.234406187773e+03
2.0416 7.468998900460e-03 1.179628855821e+03
2.1140 7.184617684986e-03 1.100429447336e+03
2.1581 7.020891385574e-03 1.055646622295e+03
2.2066 6.848400337779e-03 1.009363958888e+03
2.2609 6.663874395715e-03 9.608833084719e+02
2.3157 6.486779405975e-03 9.154830188656e+02
2.3700 6.319634002840e-03 8.735396132768e+02
2.4247 6.158784440460e-03 8.342622259708e+02
2.4519 6.081569440010e-03 8.158112663524e+02
2.5090 5.924597747320e-03 7.789272740644e+02
2.5500 5.816478973920e-03 7.537613044828e+02
2.6100 5.664482000550e-03 7.242010816107e+02
2.6700 5.519170096890e-03 6.864734956413e+02
2.7021 5.444167606810e-03 6.699569342458e+02
2.7621 5.308784127550e-03 6.404425346584e+02
2.7921 5.243268694090e-03 6.263865953175e+02
2.8522 5.116593407130e-03 5.996432134764e+02
2.9181 4.983713588500e-03 5.721914842159e+02
2.9511 4.919314244020e-03 5.589940552769e+02
3.0812 4.679970512870e-03 5.106969944804e+02
3.1481 4.564968446760e-03 4.880077821408e+02
3.2146 4.455649870800e-03 4.668957098317e+02
3.2838 4.346867783740e-03 4.462480349431e+02
3.3563 4.238114525650e-03 4.259906616078e+02
3.3924 4.185649135570e-03 4.163653275511e+02
3.4596 4.091324822560e-03 3.992125008020e+02
3.5279 3.999509364830e-03 3.827166554260e+02
3.6027 3.903127804290e-03 3.656547288210e+02
3.6781 3.810335320290e-03 3.495763860122e+02
3.7527 3.722479350320e-03 3.346334068853e+02
3.7900 3.679965743060e-03 3.274841826735e+02
1B. 1024^3 data
This data used a deterministic forcing in the first two wave number
shells, as described in (Taylor, Kurien, Eyink, Phys. Rev. E. 68, 2003).
The run is initialized with random uncorrelated phases and a prescribed
k^-(5/3) energy spectrum. The data reaches a statistical equilibrium
around t=1.0, and most of the snapshots are from after this time, but
we have also included snapshots from t=.5 and t=.8.
Duration: t=0 to 2.5
R_lambda = 460 +/- 5% for t>0.5
Eddy Turnover time: 1.05
This data is stored on disk2 in the "sc1024A" directory.
Filenames are of the form:
sc1024A????.????.uc
sc1024A????.????.vc
sc1024A????.????.wc
where ????.???? is the simulation time.
The simulation was initialized with a k^-5/3 spectrum
(random phase) at t=0, and the run to time t=2.5.
snapshots data:
time ke enstrophy
2.5000 0.18685664640632E+01 0.99769437679933E+05
2.4000 0.18733191607508E+01 0.10420300848982E+06
2.3000 0.18989228578210E+01 0.10805798457487E+06
2.2000 0.19219437577668E+01 0.10958010829487E+06
2.1000 0.19316863765580E+01 0.10538607005809E+06
2.0000 0.19241592622061E+01 0.10185734802712E+06
1.9000 0.18979338733753E+01 0.99940630814358E+05
1.8000 0.18579778733417E+01 0.99127252949821E+05
1.7000 0.18543735272404E+01 0.10126378248862E+06
1.5000 0.18731786732484E+01 0.98521452026956E+05
1.4000 0.18804331989756E+01 0.10025657478789E+06
1.3000 0.18659376839902E+01 0.10391080424108E+06
1.2000 0.18905897872825E+01 0.10823996500447E+06
1.1000 0.19165399498552E+01 0.11063981923013E+06
1.0000 0.19599579614381E+01 0.11634538084080E+06
0.8000 0.19628859030721E+01 0.10952098993512E+06
0.5000 0.19703730180068E+01 0.13790919733085E+06
2. Grid Space Compression Algorithm
The data sets described above are stored in a grid space version of
Bob Kerr's compression algorithm.
The compression involves two steps.
Step 1: Fourier Filtering
For the 2048^3 data (wave numbers 0..1024) the data was first Fourier
filtered by removing wave numbers beyond 720. Then an FFT was used to
transform this data back to grid space, but now on a 1440^3 grid.
We chose 720 because this is slightly more than a 2/3 dealiasing
(which would truncate to 683), and 720 corresponds to an FFT of length
1440, which can be handled by any FFT that can do powers of 2,3 and
5. (2^5 3^2 5).
The 1024^3 data set was not filtered.
Step 2: Utilize the fact that the flow divergence free
We then output the grid space values of the first two velocity
components, U and V, as a real*4 "brick-of-floats". For the third
velocity component, W, we only about its average in the z-direction,
since the rest of that field can be obtained from the fact that the
flow is divergence free. This data is again the grid space values, on
a standard "square-of-floats". The output is headerless, in IEEE
single precision (little endian) format, and written out in the
natural x,y,z order.
Note that all steps were performed in double precision (real*8), but
the final output was truncated to single precision (real*4). Thus for
each snapshot of the 2048^3 decaying data, we have three files:
file.u 1440^3 real*4 numbers = u component of velocity,
on a 1440^3 grid
file.v 1440^3 real*4 numbers = v component of velocity
on a 1440^3 grid
file.w 1440^2 real*4 numbers = z average of w component of velocity
on a 1440^2 grid
for a total of 23887877768 bytes (22.25GB) per snapshot.
For each snapshot of the 1024^3 forced dataset, we have two files
of size 1024^3 and one file of size 1024^2, for a total of 8GB
per snapshot. The data was computed and compressed on
LANL's QSC system (A Compaq ES-45 cluster).
To uncompress the data, we use the divergence free relation,
in the Fourier/frequency domain. Let (uf,vf,wf) denote
arrays of complex Fourier coefficients for the flow (u,v,w),
with wave numbers (i,j,k). Because of
sqrt(-1)*i*uf + sqrt(-1)*j*vf + sqrt(-1)*k*wf = 0
we can recover all Fourier coefficients of wf from uf and vf, except
for k=0. The contribution to w from the k=0 coefficients is of course
the z-average, which is exactly what is stored in "file.w".
3. Pseudo Code to Uncompress the Data:
Here is some pseudo code which shows how to read the data and recover
the w component by taking FFTs. Running this or a similar code would
require 135GB of memory. It is hoped that this pseudo code will
show how one can modify an existing spectral DNS code to be able
to read an uncompress the data. Or contact Mark Taylor for
more details or for a copy of the TWG DNS code (Fortran/MPI) which can
read and uncompress this data on any cluster.
integer(parameter) :: N=1440 ! 2^5 * 3^2 * 5
integer(parameter) :: Nf=N/2 ! number fourier coefficients
real*8 :: u(N,N,N),v(N,N,N),w(N,N,N)
complex*16 :: uf(0:Nf,0:Nf,0:Nf),vf(0:Nf,0:Nf,0:Nf),wf(0:Nf,0:Nf,0:Nf)
call mpi_open(fidu,filename.u)
call mpi_open(fidv,filename.v)
call mpi_open(fidw,filename.w)
do k=1,N
do j=1,N
do i=1,N
call mpi_read(fidu,u(i,j,k))
call mpi_read(fidv,v(i,j,k))
enddo
enddo
enddo
do j=1,N
do i=1,N
read w(i,j,1)
do k=2,N
w(i,j,k)=w(i,j,1) ! fill in w everywhere with its z-average
enddo
enddo
enddo
!
! at this point, w only contains the field averaged in z.
! now *add* in the rest of the field, using div(u,v,w)=0
!
call fft(u,uf,N)
call fft(v,vf,N)
call fft(w,wf,N)
do k=0,Nf
do j=0,Nf
do i=0,Nf
! divergence free implies that
! sqrt(-1)*i*uf + sqrt(-1)*j*vf + sqrt(-1)*k*wf = 0
! solve for wf to get:
if (k/=0) then
wf(i,j,k) = wf(i,j,k) - (i*uf(i,j,k) + j*vf(i,j,k)) / k
endif
enddo
enddo
enddo
! transform back to grid space:
call ifft(uf,u,N)
call ifft(vf,v,N)
call ifft(wf,w,N)