Changing the mxrooms parameter

[AleksanderKamienski]

Hi, I would like to compile CFAST to be able to perform simulations for a building with more than 100 rooms. I am using Ubuntu 22.04, gfortran compiler. I set the mxrooms parameter in the cfast_parameters.f90 file to 751 and I get the following error:
"
solve.o: in function __solve_routines_MOD_solve_simulation': solve.f90:(.text+0x5997): relocation truncated to fit: R_X86_64_PC32 against .bss'
solve.f90:(.text+0x59c5): relocation truncated to fit: R_X86_64_PC32 against .bss' solve.f90:(.text+0x59f3): relocation truncated to fit: R_X86_64_PC32 against .bss'
solve.f90:(.text+0x5a21): relocation truncated to fit: R_X86_64_PC32 against .bss' solve.f90:(.text+0x5a4f): relocation truncated to fit: R_X86_64_PC32 against .bss'
solve.f90:(.text+0x5a7d): relocation truncated to fit: R_X86_64_PC32 against .bss' solve.f90:(.text+0x5aab): relocation truncated to fit: R_X86_64_PC32 against .bss'
solve.f90:(.text+0x5ad9): relocation truncated to fit: R_X86_64_PC32 against .bss' solve.f90:(.text+0x5b16): relocation truncated to fit: R_X86_64_PC32 against .bss'
solve.f90:(.text+0x5b44): relocation truncated to fit: R_X86_64_PC32 against `.bss'
solve.f90:(.text+0x5b72): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
make: *** [../makefile:110: gnu_linux_64] Error 1
"

I noticed that the limit value for mxrooms is 174. If I set mxrooms below this value, the program compiles. If I set mxrooms above 174 it displays the above mentioned error or a slightly different one.

I also had an error when I tried to compile on an Intel compiler on Windows using Visual Studio. Then the maximum value of mxrooms for which the compiler compiled into the .exe file was 171. When I set a higher value, an error was displayed:

"
Severity Code Description Project File Line Suppression State Details
Error fatal error LNK1248: image size (80A64000) exceeds maximum allowable size (80000000) LINK
"

Do you know how to solve these problems so that I can compile for mxrooms set to for example 751 and the program will work properly?

Thank you in advance for your help and best regards

[preneke]

The short answer is that you don't have enough memory allocated to the linker to compile the code, note that the last error code is for, if I am reading it correctly, 80 MB. The slightly longer answer will take a bit of digging to find how to increase the size of the stack, I believe, it has been a while since we had to worry about that. Are you sure you need 751 rooms. While that maybe the number of compartments in the building unless the original fire source is a nuclear bomb or a hell mouth opening it is very unlikely that rooms that are sufficiently far from the fire will have see any significant impact. While you wait for the compiler directives to try and compile that big a version let me suggest that you create a case that has the fire room and every room that is say 4 connections away from the fire room. I would even suggest that compartment that are more than 2 connections away from the fire be treated as single zone rooms. The case will run much faster. Connections to rooms even further from the fire should just be open to the outside. If the temperature rise in the furthest rooms are less that 5 to 10 C you don't need thr whole building. You might want 5 rooms away but that is about it.

[preneke]

In cfast\Build\CFAST\makefile there is a line in #64 bit section under # ---------Windows---- that sets /F1000000000. Given your error message from building in VS it implies you are not building the 64 bit version. Can you verify that you are building the 64 bit version? Thanks.

[AleksanderKamienski]

Thank you for your quick reply.

Now I set the 64 bit version in Intel Fortran on Windows, it still doesn't want to build, the following information is displayed in the "output" window of Visual Studio:

x64\x64\solve.obj: catastrophic error: Variable SOLVE_ROUTINES_mp_INITIAL_SOLUTION$WORK.0.6 too large for NTCOFF. Bigger than 2GB. Use heap instead compilation aborted for C:\Users\Alek\Desktop\cfast-master (1)\cfast-master\Source\CFAST\solve.f90 (code 1)

The problem is with exceeding 2GB of memory, I read on the Intel forum and did not find a solution to increase the available memory above 2GB.
At this point, I have put aside the Intel compiler and I am trying to use the gfortran compiler but, as you saw in the previous message, there are also problems there.

Regarding your suggestion to reduce the number of rooms for CFAST simulation to only those that are within 2-4 connections of the fire room, this is an interesting proposal that we will consider.

In our case, we perform risk analysis based on many simulations, several hundred or even several thousand. Fires are randomly placed in different places in the building. The building geometry in our application is defined by the user and may be very different depending on the building for which we want to analyze. It would be difficult to write the appropriate code that will automaticaly select the "important" rooms for every fire scenario and every building geometry from the point of view of CFAST fire development simulation, so that the fire simulation accuracy is good enough for every scenario. It would be easier for us to have a compiled version of cfast that will work for a large number of rooms, 750 or even 1000 taking into account that the simulations will take a little longer, but be sure that the accuracy of the CFAST fire simulation will be high enough each time.

[preneke]

I understand your problem but you need to understand that CFAST won't take a little bit longer. It will take a lot longer, especially if you use the two zone rooms everywhere. Part of the problem is that the two zone model is not a correct model in rooms with very little temperature gradient. It introduces noise and things tend to slow way down. To give some perspective, 1000 rooms means 4000 equations for the solver and the Jacobian matrix is 4000 x 4000 or 16,000,000 values all being processed simultaneously, i.e. the space needed is on the order of at lease the square of the number of rooms. A huge number of those values should be zero but noise. I think you will find it faster and more usable, including reducing the rather massive output of CFAST, which scales on the order of like 20 times the number of rooms used, to break the buildings into zones and do the analysis that way.

Also, I am not an expert with the gfortran compiler and my go to expert is on vacation. I will see what I can do to assist but I expect the size issue is likely to arise no matter the compiler.

[AleksanderKamienski]

Thank you very much for your answer.

It seems that dividing the rooms into those "relevant" from the point of view of the CFAST fire simulation and performing a simulation including only these rooms can be a solution given the information you provided.

In our application, we perform many simultaneous simulations of evacuation and fire development in a building. People moving in the building (evacuating) inhale fire gases (fractional effective dose coefficient - FED). Based on the statistical data obtained, we assess the fire safety of the building.

I wonder if you have any good practice or instructions as to which rooms to choose for CFAST fire simulation if you have a large building with, say, 500 or 1000 rooms, so that the accuracy of the calculated fire simulation is good enough compared to the longer simulation performed on CFAST compiled for mxrooms = 1000 rooms.

In our case, we want the results to be accurate, so that the values ​​of the FED coefficients, temperature and visibility range are as close to the real ones as possible. We care more about the accuracy of the results than about reducing the CFAST fire simulation time.

I came up with this solution to the problem:
Assume rooms that are "important" from the point of view of the CFAST simulation as the room affected by the fire and rooms within 1 and 2 door connections. Then perform a CFAST simulation and check whether in any of the rooms 2 connections away the increase in FED, visibility range or temperature exceeded certain limit values ​​(for example for a temperature of 5 °C) and if it exceeded it, repeat the CFAST simulation but include in the next simulation the rooms following the rooms in which the limit values ​​were exceeded. And so repeat the CFAST simulation until the FED, visibility range or temperature increases do not exceed the defined limit values ​​in any room furthest from the fire source - the last CFAST fire simulation will be a sufficiently accurate simulation :)

The above solution is a bit complicated, if you managed to find information on how to easily compile cfast on the gfortran compiler for mxrooms =751 or 1001, we would be grateful for the tip.

[rmcdermo]

@mcgratta When I compile with gnu_osx_64_db after setting mxrooms=1001, I get this error:

6fdb5057-dirty\" -DGITDATE_PP=\""Mon Mar 18 19:47:54 2024 -0400\""  -DVERSION_PP=\""Test\"" -DBUILDDATE_PP=\""May 15, 2024  07:01:05\"" ../../../Source/CFAST/solve.f90
../../../Source/CFAST/solve.f90:71:43:

   71 |     integer, parameter :: lrwork = (3*mxalg**2+13*mxalg)/2
      |                                           1
Error: Result of exponentiation at (1) exceeds the range of INTEGER(4)
../../../Source/CFAST/solve.f90:223:60:

  223 |     integer, parameter :: lrwork = 40+(maxord+4)*maxeq+maxeq**2
      |                                                            1
Error: Result of exponentiation at (1) exceeds the range of INTEGER(4)
make: *** [solve.o] Error 1

I get the feeling that we need to rework some things to first count the number of rooms before assigning these parameters. I don't think it is unreasonable to use CFAST for more than 100 rooms. But correct me if I am misunderstanding what this parameters means.

[drjfloyd]

@AleksanderKamienski

A room ony matters if there is a flowpath from it to the fire that exists for enough time for there to be significant air exchange. In a building with 1000 rooms, I would expect that many of those would not have a continuous flow path between them - they are on different floors, are not normally occupied spaces and would have doors closed, are part of a different wing of the building and would be isolated by a fire door early during a fire, etc.

Depending on your budget, another option is FSSIM a commercial tool offered by Jensen Hughes. It is a fire network model where you can treat rooms as one or layer compartments and it does not have a predefefined maximum number of rooms and it uses a different solution technique that takes advantage of sparsity. However, like CFAST the more rooms that are connected by flowpaths you have the more time and memory the solution will require.

@preneke

Why is mxrooms defined prior to runtime? You could count the number of rooms in the input file and set mxrooms that way. You have a bunch of integer parameters that use mxrooms, but those could also just be set at runtime as a subroutine rather than as a parameter. For a very large number of rooms you could warn the user of potentially large run times.

An additional technique is to investigate the geometry for hydraulically seperate regions. Those do not need to be solved in the same matrix.

[drjfloyd]

@rmcdermo not just count the number of room but count items in rooms as well. That mxalg contains parameters that are functions of mxrooms and some maxium allowed number of items per room and most rooms will have a handful of items at most (vent, target, fire, etc.).

[preneke]

The truthful answer is that CFAST is a Fortran77 program. With in some limits we could do as you suggest. I don't have a lot of experience, i.e., none, with programs that dynamically determine their memory requirements. Like, I have no idea how needing more memory than the machine has is handled.

The input would have to be restructured. I am always leery of estimating time since programing is famously difficult to do that for but I don't think it is a huge job. I think all we have to do is go through the file once counting namelists and set up memory and go through it again. Plenty of things would have to be changed so it might have surprises along the way. It would not be a trivial exercise.

I want to stress that doing this doesn't solve the problem of CFAST slowing down with huge buildings. Also while CFAST solver is not that efficient with memory, my concern about rooms that are not relevant to solution is that the random noise has an amazing ability to synchronize and start adding constructively driving the solution time step down.

Aleksander, there isn't a simple rule of thumb for determining the size of the subset you need. My suggesting is pick a small set of connected rooms you want to do Monte Carlo simulations for. Include all rooms that are some number of connections away, 4 probably works. Make rooms that are more than two connections away 1 zone rooms. I don't know why but they seem to run more that twice as fast. Run a small number of cases with the largest fires you would want to run that go for as long as you want to let them and see if the single zone rooms have a significant elevation in temp. If they do add rooms that are further away and make all the rooms that are the same distance in connections two zone rooms. I like, but it is just preference, to have two layers or rooms, i.e., in this case 3 and 4 connections away, as 1 zone rooms as kind of a buffer to represent the rest of the building.

Let me also make a plug for CData. It is part of the CFAST package and it is designed to generate a set of Monte Carlo cases that can be run in a batch or script to give you a large set of data to do your analysis from.

[preneke]

@rmcdermo and @drjfloyd the lines that Randy included are calculating the work space for DASSL. That is just an FYI.

The big problem with including even 100 rooms is that we don't have data to test against. How good is CFAST with 1000 rooms. I have no idea. I tend to think it will test out as well as CFAST with the kind of data we test against but I have no idea. You know as well as I that all models have their limits and 1000 rooms may be past CFAST's, no one knows.

[rpeacoc]

Actually, we do allocate a number of variables in the initialization routine. It might be possible to improve the setting of the size of the solver work arrays, but they still depend on the square of the number of equations.

[preneke]

Yes, sorry. I should have said that more carefully. We allocate variables, just to fixed values and as I said I don't think it is a huge job to make the change. I am mostly worried about the order of variables being allocated and that kind of thing. I don't know of any hidden traps and those are always the ones that get you. The infamous "things you don't know you don't know."

[rmcdermo]

I would think that a good research topic for CFAST would be to improve its computational efficiency for relatively large numbers of rooms. This probably involves reworking the ODE solver to use a sparse matrix, i.e., not allocating N^2 for the matrix. This may not be trivial, but it is very doable, and pretty much standard practice. I think the right move would be to eventually convert CFAST over to CVODE, which we are exploring right now in FDS and which has GPU support.

I think that reasonable verification tests could be developed to give us confidence that CFAST is working well for large numbers of rooms, even if we do not have direct data.

[preneke]

Obviously I am not an expert on CVODE. I am also not married to DASSL. However, DASSL is used to solve algebraic equations in the same system as the ODEs. I believe one place algebraic equations are used is the connection of conduction into surfaces and the compartment. Also, maybe this can be worked around, while the Jacobian is generally sparse, it is not required to be banded. Suppose there is an atrium with a number of floors of rooms connected to it. Changes in each room will impact the atrium directly and vice versa. However, the rooms will not directly impact each other. It is not exactly true but you can imagine the Jacobian having the first row and column be dense and a banded diagonal, assuming the atrium is the first room. There is no way to eliminate that impact and it is very important to the answers. I do think that it probably is time to look at a new solver, I just don't want to assume the answer.

While the solver is not unimportant, I think that the physics is a bigger issue. For instance, in reports of fires in hotels I have always wondered how much of the smoke in the distant rooms and floors was driven by pressure and temperature differences and how much by diffusion. CFAST doesn't have a diffusion model. Does that impact the answers in the far rooms? I don't know. One of the issues in the issue tracker is the assumption that the ceiling jet keeps going to the wall. In big rooms that is likely not true and isn't true for the model we have. There isn't an obvious fix for this and there doesn't seem to be any data to use. We maybe able to put in something that eliminates the obvious impact of the problem but like conduction between rooms, we don't know if we fixed the problem and likely other problems arise.

My point is not to be a Debby Downer. I just don't think a lot of these problems are amenable to simple fixes without at least some data to justify the answer.

[preneke]

I should have reiterated in my last comment. I am not opposed to making CFAST able to run a 1000 rooms on a computer with enough resources or allocating memory based on the case. It maybe be easy. It seems it would be relatively easy but sometimes those are the hardest cases and I don't have a lot of confidence in predicting which is which. I understand Aleksander's desire to enter the building once. I am just concerned with unintended consequences and calculation errors that we can't anticipate.