Hi Michael,

I`m just testing a merge operation from a 1.5 Gb ECW and everything seems to be working fine, the memory having stabilize at around 1.4 Gb (from the 500 Mb it was at the start) and the merge process still going on after 15 minutes or so.

Note that even if you have 8 Gb of RAM Descartes can only uses 4 Gb on a 64 bit OS or 2 Gb on a 32 bits OS since Descartes is 32 bits.

Also even if there is 1.8 Gb or RAM available for your process the memory might be fragmented so that big chunk of memory cannot be allocated.

My advice would be to first try to lower the 2Gb. If you attach your ECW in a new .DGN the memory required would likely be much less than 1GB before starting the merge. Also you could try merging smaller zones, then merge those smaller zones together. Note if using that strategy I would recommend using a lossless compression for the temporary smaller zones as to not decrease too much the quality of the pixels.

Thanks,

Mathieu

Hi Jason and Mathieu,

Thanks for the replies and ideas.

Jason: I tried the corridor option - that crashed at 36%.

Mathieu:

I closed all applications before launching Descartes.

Before launch, the system was using 1.39gb of memory.

When Decartes was open, with the large raster attached, it was using 1.73gb.

I opened the Merge tool, selected the clipping shape, and the dialog appears - now using 1.81gb.

I click the process button (or save... or whatever it is). Within 30 seconds, Descartes and MicroStation have crashed - so that I am now looking at my desktop (no crash warnings or messages) - maximum memory usage was 1.83gb.

If I change the output resolution from 5cm to 20cm, and the clip succeeds.

So I then divided the required area into 4... as suggested, while keeping the 5cm resolution.

Again, within 30 seconds, I’m looking at my desktop.

So I then tried this same approach at 10cm resolution (area broken into 4). This time I was looking at my desktop within 5 seconds.

Mathieu - any ideas on why?

Hello Michael,

Are you only processing raster data or are you trying to include vector data in the merge?

You could also try using the Corridor tool in Descartes.  It is found in Raster Manager under the "edit" heading.  The way the tool works is you draw a closed element and the tool uses that element to capture what is inside of the element and generates a new image.  ECW is an available output image format for this tool.  You can take a look in the Descartes help to see an explanation of all the settings in the Corridor dialog to set what is best for your desired output.

Regards,

Jason Gingrich

Hi Michael,

I remember once we had a client who sent us some corrupted ECW. The ECW could be seen at lower resolution, but when seen at higher resolution at a particular location (seem the corruption was localized!) the system crashed.

So corruption maybe the cause.

Also when merging to ECW we create a temporary iTiff file (e.g. : C:\Users\Mathieu.St-Pierre\AppData\Local\Temp\dc266.iTIFF) because ECW doesn't support editing.

So maybe your C: drive doesn't enough space? So another test would be to merge to iTiff instead of ECW.

Finally it would be interesting to check if it still doesn't work if a very small area ( few hundreds by a few hundreds pixels shown for the Number of Pixels column on the Merge Options dialog).

If the problem still persists and you are willing to send your dataset to us, I'll be willing to check it further.

Thanks,

Mathieu

Have you tried setting your output image format to .ITIFF64?  ECW raster images format is compressed and creates an extremely large out put file when you are using a different raster image format.  The .iTIFF64 raster image format should handle the output file size that you are trying to create.

Thanks Mathieu and fmcallis for your suggestions.

Apologies for the delayed response - I was out of office last week.

I tried the latest suggestions. Unfortunately, all failed.

Mathieu... I'm wondering if the ECW is corrupt; as you suggested. The scenario you outline with another Client does indeed seem to fit the pattern I am seeing.

It confuses me that I should be able to generate lower resolution results, but not the same resolution as the original. If the file is corrupt, wouldn't it always fail? I guess not.

I was attempting your suggestion of a very small area to clip. I zoom in to a small area, and MicroStation/Descartes crashes. And it crashes every time I attempt to zoom in to a small area. It does the same when in MicroStation only. Quite odd.

I have 150gb of available disk space - so that should be OK.

The ECW is 13gb. I really don't know how I'd get it to you.

Is there any available software (that you are aware of) that can examine the ECW and report corruptions?

Thanks again for the suggestions.

Hi Michael,

Well, the corruption case I`m aware of was occurring only in a particular region, so zooming to the full resolution anywhere but to this particular region worked.

I have never seen a 13Gb ECW file, but I guess it should work.

If you cannot zoom to the full resolution in plain MicroStation that means the problem is not the Merge tool, but the code that is handling the reading of the ECW.

The best third party software to try would be ErViewer (see download.intergraph.com), which is the official free viewer developed by the company creating the ECW file format.

Oh we are a bit lagging to the third party library we are using in V8i. We have updated it for the update it for the MicroStation 64 bits whose EAP1 should be available soon, so maybe that would also be worth checking (I would be happy to check it, but as you said 13Gb ECW is a big extreme for uploading to our FTP site).

By the way did you send your crash dump occurring during the zooming? I`m not seeing any when searching for Gilham.

Thanks,

Mathieu

Hi Mathieu,

I'll download the viewer and see if it generates any issues/messages.

There is no crash dump; it happens far too quickly.

One second I am using the mouse wheel to zoom in, the next millisecond I'm looking at my desktop :-)

Thanks again for the reply,

Michael