Smoothing two-stack microstructure - jutting nodes after smoothing

[sgbaird]

Hi @siddharth-maddali,

Thanks for your help so far. I've been plotting the grains after doing h-smoothing and have been getting some strange results. See for example:

I also get:

109030 of 800612 nodes not smoothed.

which may be related to the quad points being too close together as you mentioned before, or it could be an issue of having a thickness of only 2 voxels in the z-dimension (I have EBSD data from both sides of a sample with a large proportion of through-thickness grains).

Using the sample data produces expected results:

If I apply DREAM.3D's laplacian smoothing filter before doing the hierarchical smoothing, I get the following, where on further inspection, it seems the triangles that "jut" leave concave "divots" in the grain.

with the following workspace:

Name                   Size                   Bytes  Class                        Attributes

  GrainToPlot            1x1                        8  double                                 
  fl               1438608x2                 23017728  double                                 
  ngrains             3348x1                    26784  double                                 
  s1                     1x1                        8  matlab.graphics.axis.Axes              
  s2                     1x1                        8  matlab.graphics.axis.Axes              
  tri              1438608x3                 34526592  double                                 
  trisub               852x3                    20448  double                                 
  xdat                   3x800607            19214568  double                                 
  xsmooth                3x800607            19214568  double     

One round-about way I might deal with this is turning the 2-stack into a 4-stack by copying the stacks above and below as appropriate and cropping it afterwards, but I'm worried this would have the artificial effect of the grain boundary always being normal to the surface when it's close to the surface.

Another similar approach would be breaking up the interior into multiple voxels, though I'm not sure how I would go about this.

Do you think this is an inherent limitation of the method for this unique use case?

[siddharth-maddali]

@sgbaird it's difficult to tell without more information on your volume. How are the nodes labeled in your example? What are the ntype labels of the nodes that "shoot out into space" after you run the smoothing?

This might have something to do with what you're seeing. h-smooth believes whatever mesh you give it in the tri variable without checking to see if the topology is messed up. I suspect some of the "jutting" nodes are labeled as interior points (2 or 12), which have the greatest freedom to move around in space.

Since you're dealing with a 3-layer (2-voxel) structure, you should try this: artificially re-label the nodes on the "rim" in all 3 layers as a triple line (3, 13), possibly bound by quad points (4, 14) if known in advance, and then run h-smooth. By this I mean use a new, custom ntype array. Note that the interior nodes in each layer can be labeled 2/12 as usual, as long as they're surrounded in the plane by nodes of type 3/13 or 4/14. If you know exactly which points along your rim are of type 4 (quad points), you should add that in artificially. Doing this hack will effectively tell h-smooth that your volume has 2 grains stacked one on top of the other, each of single-voxel thickness.

Back in the day I didn't bother adding a 2D smooth capability to h-smooth, because you can easily "hack" the 3D smooth for 2D meshes with the correct re-labeling of the nodes. Proper smoothing really depends on the tri-mesh you give it (remember, this remains unchanged before and after smoothing). If the mesh was defined poorly, the smoothing will proceed unpredictably. This problem goes back to DEREAM.3D's QuickMesh, which behaves funny sometimes.

Let me know if this hack works...

[sgbaird]

Hi @siddharth-maddali,

Thanks for your suggestions!
I modified the Test function to plot the node types (see attached)
run.txt
Test.txt

Isoview:

Sideview:

Another example:

Based on this it seems pretty apparent that jutting nodes are interior nodes (blue) as you suspected.
Partially for my own reference and partially for others, I'm copying the node definitions here:

Id Value	Node Type
2	Normal Vertex
3	Triple Line
4	Quadruple Point
12	Normal Vertex on the outer surface
13	Triple Line on the outer surface
14	Quadruple Point on the outer surface

Also, since the first grain was a pretty good candidate to show what I'm trying to accomplish on a higher level, here is the side-view "interpolation" that I'm essentially hoping to get in the end by whatever means necessary:

One thing I didn't explain (or I may not have understood your comment about 2 grains being stacked on top of each other) is that the top and bottom layers should generally be the same grain (i.e. a columnar or "through-thickness" grain).
If h-smooth interprets the volume as "2 grains stacked one on top of the other, each of single-voxel thickness", would it still tend towards the interpolation I'm hoping to accomplish above?

[sgbaird]

Also, to give a little more context, DREAM.3D's Laplacian smoothing filter starts to do what I'm hoping for, but despite trying different parameters the outer surfaces either always retained their "blocky" appearance or made strange distortions to the outer surfaces (latter not shown here). This (and a post on the DREAM.3D google groups forum) were what originally led me to look at using h-smooth.
Isoview (hsmoothed version on right):

Topview:

[siddharth-maddali]

Hi @sgbaird ,
I think I understand your data a little better now. Based on the side view you've shown, I take back what I said earlier about pretending to have two grains, one stacked on the other. That was the wrong visual to have it seems, sorry for that!

There might be a problem because of the size of the Z-axis separation of the layers, relative to the mesh size in the XY plane (assuming the isosurfaces you've shown have aspect ratio 1:1:1). You might try artifically compressing the Z-dimension of the grain to be comparable to the size of the XY discretization (i.e., the size of the mesh element). This will involve changing the Z components of your nodes. Then attempt the smoothing, and finally apply the inverse Z-scaling operation on the smooth nodes. By doing this, you might stop the nodes jutting out like that. I can't guarantee that you'll get the smooth variation from layer to layer that you're looking for, but you might reduce the "steppiness".

If the nodes still jut out, then I'm out of ideas! Other than blaming it on possibly poor meshing, because I know for sure that if the mesh is topologically sound, h-smooth will work! I am, however, curious. Can you share a data file for that particular grain (nodes, tri, ntype, fl) that I can load in Matlab? I'm quite busy these days, but I'll play with it in my free time and see what I can glean from it.

[imikejackson]

Just to comment that we started to integrate the H-smooth into DREAM3D here but ran into problems that stem from a commit that we put into the QuickMesh to try to "fix" some bad nodes (Commit 1ed41586a0bf7e9c528a11dd65c2530aa1d7682d) which I think we got as far as figuring out that this caused some of the issues with the H-Smooth algorithm. Unfortunately I don't have any kind of timeline as to when the code will be completed.

[sgbaird]

Apparently @jercgreen6 ran into some problems with nodes jutting out, even though the microstructure he was using is fairly straightforward (cube with "normal" looking grains). Additionally, he mentioned that he's only seeing the issue with the C++ code, but not the native (slow) MATLAB version. @jercgreen6, could you comment more on this?

[jercgreen6]

Initially I set up the MATLAB version of Hierarchical Smooth to verify that the code would do what I wanted it to do. Other than computation time the results were ideal so I set up the C++ version. I modified my MATLAB script which I use to run Hierarchical Smooth to match the input arguments and remove the buffer elements. However, I get very different results between the two codes for the same microstructure. The C++ version produced spikes in the microstructure similar to what was shown by @sgbaird.

I also get the same warning:
WARNING xxxx of xxxx nodes not smoothed.

This has occurred for all of the microstructures that I have tried. Microstructures were produced using Dream3D.

Figure 1: Smoothed microstructures using the C++ version of the code

Figure 2: Smoothed microstructure using the MATLAB version of the code

Included in the zip folder is the data that I used as the input to Hierarchical Smooth.
M.VerticesSmooth = HierarchicalSmoothMatlab( tri, M.Vertices.', fl, nodetype).';
M15Data.zip

@siddharth-maddali Do you have any ideas on what is going wrong here? Do you have suggestions on how to address this issue?

Thanks!

[sgbaird]

@jercgreen6, nice explanation. I wonder if you get the same kind of results using ex1 and ex2 from the examples in the repository using these instructions. If it runs OK for the example microstructures with the C++ code, it makes me think maybe it's an issue with the export/conversion of labels .. Might be worth checking to see if there are any blaring differences between the text files in the example repositoyr and M15Data. Also, did you remove the buffer mesh elements on the faces of the box:

f = find( any( fl > 0, 2 ) );
tri = tri( f, : );
fl = fl( f, : );

before running it through HierarchicalSmoothMatlab()? This might not affect anything (the documentation suggests it only affects the runtime), but thought I'd check.

All-in-all a strange issue and the inconsistency between versions suggests a possible bug. How urgent is it (days, weeks, etc.)?

Also, @siddharth-maddali, sorry for not including a dataset before. Hopefully the one @jercgreen6 included will be enough to resolve the issue. Thanks in advance for the help! Let us know when you get this and what your thoughts are

[jercgreen6]

For the examples I still get the error about some of the nodes not smoothing in the C++ code (they work fine for the MATLAB code). This would suggest that it is either how I am interfacing with the code or an issue with the code itself. I am in the troubleshooting phase to try an figure out what is going on.

I do remove the buffer elements before running the code. I tried running it without removing the buffer elements and more of the nodes smoothed but there were still a large amount which did not.

A solution within 2 weeks would be ideal but not required.

[sgbaird]

@jercgreen6 do you still get the "jutting nodes" with the examples via C++?

@siddharth-maddali had mentioned to me in #9 (comment) about why some number of nodes aren't smoothed, but this didn't seem to be a major problem.

[sgbaird]

It would appear there's also a difference in indexing the triangulation between the MATLAB and C++ versions.

For the MATLAB version, based on the documentation, I think it's:
​tri = ​1​ ​+​ load( ​'​../../examples/ex2/SharedTriList.txt​'​ );

Whereas for C++ MEX version I'm guessing it's just:
​tri = load( ​'​../../examples/ex2/SharedTriList.txt​'​ );

Just trying to think through any possibilities other than an error in the C++ code which will probably be more painful to figure out.

[jercgreen6]

@sgbaird
So far I have been only able to observe the effect of the C++ code on the example files. The code which I use in conjunction with Hierarchical Smooth doesn't work well with the MATLAB version of Hierarchical Smooth. As far as I can tell this is due to several changes in Dream3D output files from when HierarchicalSmooth was created.

• Dream3D no longer creates microstructures containing buffer elements. These are not considered when using Hierarchical Smooth but buffer elements need to be removed from the dataset to process correctly with my code.
• Dream3D doesn't distinguish a grain neighboring empty space between the sides and top (only uses -1, no 0s). 0's need to be replaced by -1's.
• With the MATLAB code I am encountering an unknown issue due to the existence of several grains having grainID 0 (breaks MATLAB's indexing rules). I tried adding 1 to the entire array to just to see what would happen but it still wouldn't go through.

As far as your comments on indexing being the issue. I tried running the code with and without adding 1 to the tri variable. The results are shown here:

Figure 1: Example 1 where 1 was not added to the tri variable. Here nodes seem to be pulled in a single direction.

Alternatively, the closest that I have been able to get to things actually working are shown here. Previously I was using the fscanf command instead of the load command. I don't know why this would make a strong difference bet it seemed to help.

Figure 2: Example1 where load was used instead of fscanf to initialize variables. Still contains the ripple effect.

At this point all that I know is that the smoothed vertices that I am getting out of Hierarchical Smooth do not match what is recorded as the example output. I'm not sure if that is to be expected using the Hierarchical Smooth algorithm. About 25% of vertex locations are the same (determined using nnz(MySmoothedVertices == ExampleSmoothVertices)) but 75% are different. @siddharth-maddali After running Hierarchical Smooth should my output files exactly match the example smooth file?

[sgbaird]

@jercgreen6, looks like you're getting closer. Assuming you're using the latest version now, perhaps try using a 2018 version of DREAM.3D? From what I'm seeing, it doesn't seem like an internal error in the C++ code so much as a version incompatibility or maybe a data processing issue. I think a MWE for the DREAM.3D output process and HierarchicalSmooth which reproduces some of the plots you've been showing will make it easier for me to do some hands-on troubleshooting (if you have time to put a MWE together). I also think it's worth asking, have you tried any of the built-in DREAM.3D smoothing filters? (esp. Laplacian).

[siddharth-maddali]

@jercgreen6 @sgbaird Thanks for letting me know. I'll get back to you as soon as I can.

[siddharth-maddali]

@jercgreen6 ,

1. I don't think the Matlab code warns you that it's ignoring nodes, even though it does in the background. The printed message is a feature of the C++ code only, to warn you that your mesh might not be resolving your microstructure finely enough in some regions of your sample. I ran the C++ code on both example data sets and only ~30 out of 159433 nodes were not smoothed. This is reasonable because the data set has some very small grains. BTW, I used the C++ code wrapped in an Octave function, but the Matlab wrapper should act no differently.

2. It appears that your input M.Vertices has already been smoothed somewhat. Are you running HierarchicalSmooth on nodes that have already been already HierarchicalSmoothed? Or smoothed by some other process? If so, it might very well misbehave the second time round. HierarchicalSmooth makes use of the rectangular pixelation to constrain the smooth surface. It finds the best minimum-energy smooth surface that can 'slide' into the thin volume defined by a sheet (or line) of pixels. If this pixelation is gone, as it will be after one round of smoothing, then HierarchicalSmooth behaves in strange ways, and that could explain the jutting nodes.

3. I usually don't bother smoothing the flat cross-sections of the grains at the faces of the box, because these are not real grain boundaries anyway. As @sgbaird mentioned above, it is usually best to ignore these nodes while smoothing, and include them later in your plot.
   

I hope this helps! I need to look into why HierarchicalSmooth sometimes misbehaves for exactly flat surfaces, as it seems to be doing.