esp_modem / PPP.cmd returns only last fragment of AT response when PPP+CMUX is active (SIM7670)

[saadmustafa12]

Board

ESP32 S3 Module N16R8

Device Description

ESP32 S3 connected with SIMCOM SIM7670G over UART 1 through PPP

Hardware Configuration

GPIO 37 & 36 I2C

Version

v3.3.5

Type

Bug

IDE Name

Arduino IDE

Operating System

macOS 15.6.1

Flash frequency

80

PSRAM enabled

yes

Upload speed

921600

Description

I am using PPP with SIM7670G modem in
PPP.mode(ESP_MODEM_MODE_CMUX);

PPP data connection works smoothly, traffic is stable, and other AT commands work correctly while PPP is active. The problem appears only when executing the GNSS command:

AT+CGNSSINFO

Observed behavior:

• When no GNSS fix is available, the modem returns:
  +CGNSSINFO: ,,,,,,,,

• When a GNSS fix is available, the modem should return a longer response such as:

+CGNSSINFO: 3,13,06,03,18,31.399003,N,73.175373,E,261225,191834.000,195.5,0.16,0.00,0.80,0.53,0.60,25
OK

However, what is actually received is only the tail fragment:

2.009,182.3,0.07,167.32,1.16,0.80,0.85,15

with the initial part (+CGNSSINFO: and earlier fields completely missing).

Key findings

The total length of the full response (including \r\nOK\r\n) is ~107 bytes, verified on ESP32 using:

• String.length()
• strlen()
• CONFIG_ESP_MODEM_C_API_STR_MAX is 128 bytes (and also tested with 256 and 512).
• CONFIG_ESP_MODEM_USE_INFLATABLE_BUFFER_IF_NEEDED is enabled.

A standalone simulation of esp_modem_at() using strlcpy() confirms:

• strlcpy() only truncates the tail, never the beginning.
• With buffers ≥128 bytes, the entire 107-byte response copies correctly.

Therefore, i think this issue is not caused by buffer size limits or strlcpy() truncation.

Important characteristics of the issue

• The same tail fragment is received consistently on every attempt.
• There is no random corruption, noise, or partial mixing of bytes.
• This strongly indicates that earlier fragments are being dropped or overwritten, and only the last fragment is returned.
• The issue happens only when the response is large.
• Short responses (including empty GNSS data) are returned correctly.

Additional testing

• I also tested without establishing a PPP data connection (no active internet traffic).
• The truncation still occurs for AT+CGNSSINFO once a GNSS fix is present.
• This suggests the issue is related to CMUX / AT response handling, not data traffic load.

This results in missing headers and partial GNSS data, even though the total response size is well within configured limits.

Thank You! Looking Forward to the solution....

Sketch

/**
 * Minimal repro: SIM76xx + PPP + CMUX => AT+CGNSSINFO returns only tail fragment
 *
 * Requirements:
 * - Arduino-ESP32 core v3.x (PPP.h available)
 * - SIM7670G (or SIM7600 family) wired over UART
 *
 * What it does:
 * 1) Starts PPP
 * 2) Switches to ESP_MODEM_MODE_CMUX
 * 3) Powers GNSS
 * 4) Polls AT+CGNSSINFO every 5s and prints length + hex + raw
 */

#include <Arduino.h>
#include <PPP.h>

// --------- Configure for your board ----------
#define PPP_MODEM_APN  "Jazz"       // <-- your APN
#define PPP_MODEM_PIN  "0000"       // or nullptr / "" if none

// UART pins to modem (ESP32 TX->MODEM RX, ESP32 RX<-MODEM TX)
#define PPP_MODEM_TX   17
#define PPP_MODEM_RX   18
#define PPP_MODEM_RTS  -1           // set if you use HW flow control
#define PPP_MODEM_CTS  -1
#define PPP_MODEM_FC   ESP_MODEM_FLOW_CONTROL_NONE

// Optional reset pin
#define PPP_MODEM_RST  7
#define PPP_MODEM_RST_LOW false     // set true if reset is active-low
#define PPP_MODEM_RST_DELAY 200

// SIM7670G uses SIM7600 driver in Arduino PPP wrapper in many builds
#define PPP_MODEM_MODEL PPP_MODEM_SIM7600
// --------------------------------------------

static unsigned long lastPoll = 0;

static void dumpHex(const String &s) {
  for (size_t i = 0; i < s.length(); i++) {
    uint8_t b = (uint8_t)s[i];
    Serial.printf("%02X ", b);
    if ((i + 1) % 16 == 0) Serial.println();
  }
  Serial.println();
}

static void runCmd(const char *cmd, int timeoutMs) {
  String resp;
  bool ok = PPP.cmd(cmd, resp, timeoutMs);

  Serial.println("--------------------------------------------------");
  Serial.printf("CMD: %s\n", cmd);
  Serial.printf("OK: %s\n", ok ? "true" : "false");
  Serial.printf("LEN: %u bytes\n", (unsigned)resp.length());
  Serial.println("RAW:");
  Serial.println(resp);
  Serial.println("HEX:");
  dumpHex(resp);
}

static void onEvent(arduino_event_id_t event, arduino_event_info_t info) {
  switch (event) {
    case ARDUINO_EVENT_PPP_START:         Serial.println("PPP Started"); break;
    case ARDUINO_EVENT_PPP_CONNECTED:     Serial.println("PPP Connected"); break;
    case ARDUINO_EVENT_PPP_GOT_IP:        Serial.println("PPP Got IP"); break;
    case ARDUINO_EVENT_PPP_LOST_IP:       Serial.println("PPP Lost IP"); break;
    case ARDUINO_EVENT_PPP_DISCONNECTED:  Serial.println("PPP Disconnected"); break;
    case ARDUINO_EVENT_PPP_STOP:          Serial.println("PPP Stopped"); break;
    default: break;
  }
}

void setup() {
  Serial.begin(115200);
  delay(800);

  Serial.println("\n=== Repro: PPP + CMUX + AT+CGNSSINFO truncation ===");

#if defined(PPP_MODEM_RST)
  pinMode(PPP_MODEM_RST, PPP_MODEM_RST_LOW ? OUTPUT_OPEN_DRAIN : OUTPUT);
  digitalWrite(PPP_MODEM_RST, PPP_MODEM_RST_LOW);
  delay(100);
  digitalWrite(PPP_MODEM_RST, !PPP_MODEM_RST_LOW);
  delay(3000);
  digitalWrite(PPP_MODEM_RST, PPP_MODEM_RST_LOW);
#endif

  Network.onEvent(onEvent);

  PPP.setApn(PPP_MODEM_APN);
  PPP.setPin(PPP_MODEM_PIN);
  PPP.setResetPin(PPP_MODEM_RST, PPP_MODEM_RST_LOW, PPP_MODEM_RST_DELAY);
  PPP.setPins(PPP_MODEM_TX, PPP_MODEM_RX, PPP_MODEM_RTS, PPP_MODEM_CTS, PPP_MODEM_FC);

  Serial.println("Starting modem/PPP...");
  PPP.begin(PPP_MODEM_MODEL);

  // Basic info (usually works)
  Serial.print("Manufacturer: "); Serial.println(PPP.cmd("AT+CGMI", 5000));
  Serial.print("Model: ");        Serial.println(PPP.moduleName());
  Serial.print("IMEI: ");         Serial.println(PPP.IMEI());

  // Wait for network attach
  Serial.print("Waiting to attach");
  for (int i = 0; i < 600 && !PPP.attached(); i++) {
    Serial.print(".");
    delay(100);
  }
  Serial.println();

  Serial.printf("Attached: %d\n", PPP.attached() ? 1 : 0);
  if (!PPP.attached()) {
    Serial.println("Not attached; stopping.");
    return;
  }

  // Switch to CMUX (this is the mode involved in the bug report)
  Serial.println("Switching to CMUX mode...");
  PPP.mode(ESP_MODEM_MODE_CMUX);

  // Ensure we are online (PPP connected)
  if (!PPP.waitStatusBits(ESP_NETIF_CONNECTED_BIT, 10000)) {
    Serial.println("Failed to get IP");
  } else {
    Serial.println("Internet OK");
  }

  // Power GNSS
  runCmd("AT+CGNSSPWR=1", 5000);

  // Optional: cold start (uncomment if needed)
  // runCmd("AT+CGPSCOLD", 5000);

  // First GNSS read (often empty until fix)
  runCmd("AT+CGNSSINFO", 10000);

  lastPoll = millis();
}

void loop() {
  // Poll every 5 seconds
  if (millis() - lastPoll >= 5000) {
    runCmd("AT+CGNSSINFO", 10000);
    lastPoll = millis();
  }
}

Debug Message

[     2][I][esp32-hal-psram.c:106] psramAddToHeap(): PSRAM added=========== Before Setup Start ===========
Chip Info:
------------------------------------------
  Model             : ESP32-S3
  Package           : 0
  Revision          : 0.02
  Cores             : 2
  CPU Frequency     : 240 MHz
  XTAL Frequency    : 4 Features Bitfield : 0x00000012
  Embedded Flash    : No
  Embedded PSRAM    : No
  2.4GHz WiFi       : Yes
  Classic BT        : No
  BT Low Energy     : Yes
  IEEE 802.15.4     : No
------------------------------------------
INTERNAL Memory Info:
------------------------------------------
  Total Size        :   384540 B ( 375.5 KB)
  Free Bytes        :   345900 B ( 337.8 KB)
  Allocated Bytes   :    33432 B (  32.6 KB)
  Minimum Free Bytes:   340752 B ( 332.8 KB)
  Largest Free Block:   286708 B ( 280.0 KB)
------------------------------------------
SPIRAM Memory Info:
------------------------------------------
  Total Size        :  2097152 B (2048.0 KB)
  Free Bytes        :  2095104 B (2046.0 KB)
  Allocated Bytes   :        0 B (   0.0 KB)
  Minimum Free Bytes:  2095104 B (2046.0 KB)
  Largest Free Block:  2064372 B (2016.0 KB)
  Bus Mode          : QSPI
------------------------------------------
Flash Info:
------------------------------------------
  Chip Size         : 16777216 B (16 MB)
  Block Size        :    65536 B (  64.0 KB)
  Sector Size       :     4096 B (   4.0 KB)
  Page Size         :      256 B (   0.2 KB)
  Bus Speed         : 80 MHz
  Flash Frequency   : 80 MHz (source: 80 MHz, divider: 1)
  Bus Mode          : QIO
------------------------------------------
Partitions Info:
------------------------------------------
                nvs : addr: 0x00009000, size:    20.0 KB, type: DATA, subtype: NVS
            otadata : addr: 0x0000E000, size:     8.0 KB, type: DATA, subtype: OTA
               app0 : addr: 0x00010000, size:  3072.0 KB, type:  APP, subtype: OTA_0
               app1 : addr: 0x00310000, size:  3072.0 KB, type:  APP, subtype: OTA_1
               ffat : addr: 0x00610000, size: 10112.0 KB, type: DATA, subtype: FAT
           coredump : addr: 0x00FF0000, size:    64.0 KB, type: DATA, subtype: COREDUMP
------------------------------------------
Software Info:
------------------------------------------
  Compile Date/Time : Dec 26 2025 04:16:46
  Compile Host OS   : macosx
  ESP-IDF Version   : v5.5.1-931-g9bb7aa84fe
  Arduino Version   : 3.3.5
------------------------------------------
Board Info:
------------------------------------------
  Arduino Board     : ESP32S3_DEV
  Arduino Variant   : esp32s3
  Arduino FQBN      : esp32:esp32:esp32s3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=cdc,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=16M,PartitionScheme=app3M_fat9M_16MB,DebugLevel=debug,PSRAM=enabled,LoopCore=1,EventsCore=1,EraseFlash=none,JTAGAdapter=default,ZigbeeMode=default
============ Before Setup End ============
[   915][I][esp32-hal-periman.c:141] perimanSetPinBus(): Pin 19 already has type USB_DM (45) with bus 0x3fc9863c
[   915][I][esp32-hal-periman.c:141] perimanSetPinBus(): Pin 20 already has type USB_DP (46) with bus 0x3fc9863c

=== Repro: PPP + CMUX + AT+CGNSSINFO truncation ===
Starting modem/PPP...
PPP Started
Manufacturer: SIMCOM INCORPORATED
Model: SIM7670G-MNGV
IMEI: 864643060739903
Waiting to attach
Attached: 1
Switching to CMUX mode...
PPP Got IP
Internet OKPPP Connected

--------------------------------------------------
CMD: AT+CGNSSPWR=1
OK: true
LEN: 2 bytes
RAW:
OK
HEX:
4F 4B 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 20 bytes
RAW:
+CGNSSINFO: ,,,,,,,,
HEX:
2B 43 47 4E 53 53 49 4E 46 4F 3A 20 2C 2C 2C 2C 
2C 2C 2C 2C 
=========== After Setup Start ============
INTERNAL Memory Info:
------------------------------------------
  Total Size        :   384540 B ( 375.5 KB)
  Free Bytes        :   321284 B ( 313.8 KB)
  Allocated Bytes   :    56944 B (  55.6 KB)
  Minimum Free Bytes:   321016 B ( 313.5 KB)
  Largest Free Block:   278516 B ( 272.0 KB)
------------------------------------------
SPIRAM Memory Info:
------------------------------------------
  Total Size        :  2097152 B (2048.0 KB)
  Free Bytes        :  2083056 B (2034.2 KB)
  Allocated Bytes   :    11664 B (  11.4 KB)
  Minimum Free Bytes:  2079712 B (2031.0 KB)
  Largest Free Block:  2064372 B (2016.0 KB)
------------------------------------------
GPIO Info:
------------------------------------------
  GPIO : BUS_TYPE[bus/unit][chan]
  --------------------------------------  
     7 : PPP_MODEM_RST
    17 : PPP_MODEM_TX
    18 : PPP_MODEM_RX
    19 : USB_DM
    20 : USB_DP
    43 : UART_TX[0]
    44 : UART_RX[0]
============ After Setup End =============
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 20 bytes
RAW:
+CGNSSINFO: ,,,,,,,,
HEX:
2B 43 47 4E 53 53 49 4E 46 4F 3A 20 2C 2C 2C 2C 
2C 2C 2C 2C 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 38 bytes
RAW:
1.263,187.6,0.40,0.00,1.49,1.15,0.95,9
HEX:
31 2E 32 36 33 2C 31 38 37 2E 36 2C 30 2E 34 30 
2C 30 2E 30 30 2C 31 2E 34 39 2C 31 2E 31 35 2C 
30 2E 39 35 2C 39 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 39 bytes
RAW:
6.000,187.6,0.05,0.00,1.26,0.87,0.91,12
HEX:
36 2E 30 30 30 2C 31 38 37 2E 36 2C 30 2E 30 35 
2C 30 2E 30 30 2C 31 2E 32 36 2C 30 2E 38 37 2C 
30 2E 39 31 2C 31 32 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 39 bytes
RAW:
1.000,187.6,0.17,0.00,1.09,0.71,0.83,19
HEX:
31 2E 30 30 30 2C 31 38 37 2E 36 2C 30 2E 31 37 
2C 30 2E 30 30 2C 31 2E 30 39 2C 30 2E 37 31 2C 
30 2E 38 33 2C 31 39 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 39 bytes
RAW:
6.000,187.6,0.01,0.00,1.04,0.66,0.80,20
HEX:
36 2E 30 30 30 2C 31 38 37 2E 36 2C 30 2E 30 31 
2C 30 2E 30 30 2C 31 2E 30 34 2C 30 2E 36 36 2C 
30 2E 38 30 2C 32 30 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 39 bytes
RAW:
1.000,187.6,0.05,0.00,0.92,0.58,0.72,24
HEX:
31 2E 30 30 30 2C 31 38 37 2E 36 2C 30 2E 30 35 
2C 30 2E 30 30 2C 30 2E 39 32 2C 30 2E 35 38 2C 
30 2E 37 32 2C 32 34 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 39 bytes
RAW:
6.000,187.6,0.08,0.00,0.92,0.58,0.72,24
HEX:
36 2E 30 30 30 2C 31 38 37 2E 36 2C 30 2E 30 38 
2C 30 2E 30 30 2C 30 2E 39 32 2C 30 2E 35 38 2C 
30 2E 37 32 2C 32 34 
--------------------------------------------------
CMD: AT+CGNSSINFO
OK: true
LEN: 39 bytes
RAW:
1.000,187.6,0.17,0.00,0.92,0.58,0.72,24
HEX:
31 2E 30 30 30 2C 31 38 37 2E 36 2C 30 2E 31 37 
2C 30 2E 30 30 2C 30 2E 39 32 2C 30 2E 35 38 2C 
30 2E 37 32 2C 32 34

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[lbernstone]

My guess is the string contains unprintable characters that the String() conversion chokes on. Can you do a hex dump of the 107 chars that you get from esp_modem_at()? Maybe a little bit of logic can be added to trim that.

[saadmustafa12]

Hello! Thanks for the response....

I modified PPP.cpp and added following:

static void hexDump128(const uint8_t *buf, size_t n) {
  for (size_t i = 0; i < n; i++) {
    Serial.printf("%02X ", buf[i]);
    if ((i + 1) % 16 == 0) Serial.println();
  }
  Serial.println();
}

bool PPPClass::cmd(const char *at_command, String &response, int timeout) {
  PPP_CMD_MODE_CHECK(false);

  char out[128];
  memset(out, 0xEE, sizeof(out));   // fill with pattern to see untouched bytes
  out[127] = 0;                     // ensure NUL at end 

  esp_err_t err = esp_modem_at(_dce, at_command, out, timeout);

  // ---- DEBUG: raw buffer inspection ----
  Serial.printf("\n[PPP.cmd] %s  err=%d (%s)\n", at_command, (int)err, esp_err_to_name(err));
  Serial.printf("FixedBuf=128, strlen(out)=%u\n", (unsigned)strlen(out));

  // find first NUL (0x00) 
  int firstNul = -1;
  for (int i = 0; i < 128; i++) {
    if (((uint8_t)out[i]) == 0x00) { firstNul = i; break; }
  }
  Serial.printf("First NUL index=%d\n", firstNul);

  Serial.println("HEX DUMP (128 bytes):");
  hexDump128((uint8_t*)out, 128);

  Serial.println("RAW PRINT:");
  Serial.println(out);
  // --------------------------------------

  response = String(out);

  if (err != ESP_OK) {
    log_e("esp_modem_at failed %d %s", err, esp_err_to_name(err));
    if (err == ESP_FAIL && response.isEmpty()) response = "ERROR";
    return false;
  }

  if (response.isEmpty()) response = "OK";
  return true;
}

But still no luck....

02:53:17.646 -> [PPP.cmd] AT+CGNSSINFO err=0 (ESP_OK)
02:53:17.646 -> FixedBuf=128, strlen(out)=41
02:53:17.646 -> First NUL index=41
02:53:17.646 -> HEX DUMP (128 bytes):
02:53:17.646 -> 37 2E 30 30 30 2C 31 38 36 2E 30 2C 30 2E 31 35
02:53:17.646 -> 2C 31 31 33 2E 30 31 2C 30 2E 37 39 2C 30 2E 35
02:53:17.646 -> 30 2C 30 2E 36 31 2C 32 37 00 EE EE EE EE EE EE
02:53:17.646 -> EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
02:53:17.646 -> EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
02:53:17.646 -> EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
02:53:17.646 -> EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE
02:53:17.646 -> EE EE EE EE EE EE EE EE EE EE EE EE EE EE EE 00
02:53:17.646 ->
02:53:17.646 -> RAW PRINT:
02:53:17.646 -> 7.000,186.0,0.15,113.01,0.79,0.50,0.61,27
02:53:17.646 -> --------------------------------------------------

[lbernstone]

Well, if esp_modem_at is giving you the bad response, I think you will need to take that up with @david-cermak at esp_modem