fix heap corruption on LP64 platforms

Mixing unsigned long and int on LP64 platforms caused the chunksize
adjustment to be wrong for flash memory reads from "negative"
addresses. This caused runaway reads and heap corruption, because
chunksize was being adjusted to be greater than numBytes. Simplify
the computation by computing the offset within the page using a mask,
and use the difference between pageSize and offset to limit chunksize.

This is less necessary after the qXfer:memory-map:read support
was added, but it's definitely needed in 2.13, and maybe some
older GDB versions don't support qXfer:memory-map:read.

Author: tlyu

src/jtag2rw.cc

@@ -151,11 +151,12 @@ uchar *jtag2::jtagRead(unsigned long addr, unsigned int numBytes)
 	unsigned int chunksize = numBytes;
 	unsigned int targetOffset = 0;
 
-	if (addr + chunksize >= pageAddr + pageSize)
+	offset = addr & mask;
+	if (chunksize > pageSize - offset) {
 	    // Chunk would cross a page boundary, reduce it
 	    // appropriately.
-	    chunksize -= (addr + numBytes - (pageAddr + pageSize));
-	offset = addr - pageAddr;
+	    chunksize = pageSize - offset;
+	}
 
 	while (numBytes > 0)
 	{

src/jtag3rw.cc

@@ -150,11 +150,12 @@ uchar *jtag3::jtagRead(unsigned long addr, unsigned int numBytes)
 	unsigned int chunksize = numBytes;
 	unsigned int targetOffset = 0;
 
-	if (addr + numBytes >= pageAddr + pageSize)
+	offset = addr & mask;
+	if (chunksize > pageSize - offset) {
 	    // Chunk would cross a page boundary, reduce it
 	    // appropriately.
-	    chunksize -= (addr + numBytes - (pageAddr + pageSize));
-	offset = addr - pageAddr;
+	    chunksize = pageSize - offset;
+	}
 
 	while (numBytes > 0)
 	{