BME680: Gas sensor heater unstable

[GArndt_vacos]

Dear members,  I am evaluating the BME680 for smoldering fire detection in a dirty environment, which means that I am looking for a way to discriminate easily and heavily oxidizable gas species. For that purpose, I have forked from Bosch's BME680 driver to make use of the sensor's ability to configure and query 10 different heater configurations (temperature and duration). You can find the modified code here: https://github.com/guarndt/BME680_driver 

I use all possible ten heater configurations; each step has a duration of 50 ms. That is more then the 20-30 ms which, according to the documentation, are required to achieve a stable heater temperature. The temperatures are 400°C down to 130°C in decrements of 30 K.

Here comes my problem: Normally, that configuration works fine. I have tested it with a BME680 on a Joy-It breakout board connected directly to a Raspi 3 via I²C at 3.3 V, and it runs flawlessly for days. I have recorded a communication cycle with an oscilloscope; see Scope_regular.png

In my lab setup driven by a Odroid C2, however, there are three of those BME680 boards connected to the I²C via a TCA9548A multiplexer and P82B96 line buffers with a supply voltage of 3.6 V. They may run equally flawlessly for up to a day, but after some (possibly shorter) time any one of them can get into an error state in which it is unable to achieve a stable heater temperature. Once it occurs, that always affects all ten temperature steps, whereas that sensor's other measurements (ambient temperature, pressure, humidity) continue to work fine. I have captured a communication cycle in the error state as well; see Scope_long.png

How to stabilize the heater? Do I have to use longer cycles? It does not seem to depend on ambient temperature, as it usually happens at room temperature.

The sensor is initialized like this:

			dev_id = BME680_I2C_ADDR_PRIMARY;
			intf = BME680_I2C_INTF;
			/* amb_temp can be set to 25 prior to configuring the gas sensor
			 * or by performing a few temperature readings without operating the gas sensor.
			 */
			amb_temp = 25;
			if (bme680_init() != BME680_OK) {
				throw new std::domain_error("Could not initialize BME680.");
			}

			/* Set the temperature, pressure and humidity settings */
			tph_sett.os_hum = BME680_OS_2X;
			tph_sett.os_pres = BME680_OS_4X;
			tph_sett.os_temp = BME680_OS_8X;
			tph_sett.filter = BME680_FILTER_SIZE_3;
			/* Set the remaining gas sensor settings and link the heating profile */
			gas_sett.run_gas = BME680_ENABLE_GAS_MEAS;
			gas_sett.nb_conv = 0;
			/* Select the power mode */
			/* Must be set before writing the sensor configuration */
			power_mode = BME680_FORCED_MODE;
			/* Set the required sensor settings needed */
			uint8_t const set_required_settings = BME680_OST_SEL | BME680_OSP_SEL | BME680_OSH_SEL | BME680_FILTER_SEL | BME680_GAS_SENSOR_SEL;
			/* Set the desired sensor configuration */
			if (bme680_set_sensor_settings(set_required_settings) != BME680_OK) {
				throw new std::domain_error("Could not configure BME680.");
			}
			/* Set the power mode */
			if (bme680_set_sensor_mode() != BME680_OK) {
				throw new std::domain_error("Could not set BME680's power mode.");
			}

Here is the code that is run in a loop to query the sensor:

for (size_t i{0}; i < gasResistance.size(); ++i) {
				if (_measurementEnd) {
					std::this_thread::sleep_until(*_measurementEnd);
					struct bme680_field_data data;
					if (bme680_get_sensor_data(&data) == BME680_OK) {
						averageTemperature += celsius_t{data.temperature / 100.0f};
						averagePressure += hectopascal_t{data.pressure / 100.0f};
						averageHumidity += data.humidity / 1000.0f;
						/* Avoid using measurements from an unstable heating setup */
						if (dayta.status & BME680_GASM_VALID_MSK) {
							gasResistance[data.gas_index]->notifyObservers(ohm_t(data.gas_resistance));
							/* Select the next gas sensor heater profile. */
							this->gas_sett.nb_conv = overflowingNext<uint8_t, BME680_HEATER_SET_COUNT-1>(data.gas_index);
							if (bme680_set_sensor_settings(BME680_NBCONV_SEL) != BME680_OK) {
								ErrorObservable::notifyObservers("Next heater profile could not be selected."s);
							}
						} else {
                        // This is the error that keeps occurring:
							ErrorObservable::notifyObservers("Measurement from unstable heating setup rejected."s);
						}
					} else {
						ErrorObservable::notifyObservers("Data could not be acquired from device."s);
					}
				} else {
					ErrorObservable::notifyObservers("Measurement had not been started."s);
				}
				/* Trigger the next measurement if you would like to read data out continuously */
				if (power_mode == BME680_FORCED_MODE) {
					if (bme680_set_sensor_mode() == BME680_OK) {
						auto const measurementStart(std::chrono::steady_clock::now());
						/* Calculate the total measurement duration (which takes time). */
						uint16_t meas_period;
						bme680_get_profile_dur(&meas_period);
						_measurementEnd = measurementStart + std::chrono::milliseconds{meas_period};
					} else {
						_measurementEnd.reset();
						ErrorObservable::notifyObservers("Could not set sensor mode after measurement."s);
					}
				}
			}

[Vincent]

I'm talking about the Scope_Odroid_error_close.jpg.  i saw there is no ACK in the end of communication.  that is the reason i asked you in previous ticket.  

I will wait for your test result with only one BME680 on the platform. 

If it works,  then maybe you can try with 3 sensors again with bit longer delay time in your sleep function. 

[GArndt_vacos]

I have continued to test the setup with 3 BME680, and later also with all other devices activated, and could for now not reproduce the problem. Longer sleep times were not required.

I have also analyzed the I²C messages decoded by the scope and found that there must have been a more severe issue, as multiple message were strange. E.g., ones that should have been the same during each cycle were incremented, which is a hint to a software bug (which I could not find, though).

However, I improved my driver code to fully evaluate BME680's error messages:

            Average<celsius_t> averageTemperature;
			Average<hectopascal_t> averagePressure;
			Average<double> averageHumidity;
			for (size_t i{0}; i < gasResistance.size(); ++i) {
				if (_measurementEnd) {
					std::this_thread::sleep_until(*_measurementEnd);
					struct bme680_field_data data;
					/* Read register 0x1D. */
					if (bme680_get_sensor_data(&data) == BME680_OK) {
						if (data.status & BME680_NEW_DATA_MSK) {
							averageTemperature += celsius_t{data.temperature / 100.0f};
							averagePressure += hectopascal_t{data.pressure / 100.0f};
							averageHumidity += data.humidity / 1000.0f;
							/* Avoid using measurements from an unstable heating setup */
							if (data.status & BME680_GASM_VALID_MSK) {
								if (data.status & BME680_HEAT_STAB_MSK) {
									gasResistance[data.gas_index]->notifyObservers(ohm_t(data.gas_resistance));
								} else {
									ErrorObservable::notifyObservers("Measurement from unstable heating setup rejected."s);
								}
							} else {
								ErrorObservable::notifyObservers("Invalid gas measurement received."s);
							}
							/* Select the next gas sensor heater profile - read and write register 0x71. */
							this->gas_sett.nb_conv = overflowingNext<uint8_t, BME680_HEATER_SET_COUNT-1>(data.gas_index);
							if (bme680_set_sensor_settings(BME680_NBCONV_SEL) != BME680_OK) {
								ErrorObservable::notifyObservers("Next heater profile could not be selected."s);
							}
						} else {
							ErrorObservable::notifyObservers("No new data available."s);
						}
					} else {
						ErrorObservable::notifyObservers("Data could not be acquired from device."s);
					}
				} else {
					ErrorObservable::notifyObservers("Measurement had not been started."s);
				}
				/* Trigger the next measurement if you would like to read data out continuously */
				/* Read and write register 0x74. */
				if (bme680_set_sensor_mode() == BME680_OK) {
					auto const measurementStart(std::chrono::steady_clock::now());
					/* Calculate the total measurement duration (which takes time). */
					uint16_t meas_period;
					bme680_get_profile_dur(&meas_period);
					_measurementEnd = measurementStart + std::chrono::milliseconds{meas_period};
				} else {
					_measurementEnd.reset();
					ErrorObservable::notifyObservers("Could not set sensor to forced mode."s);
				}
			}

[Vincent]

OK, this means 3 sensors are working properly now on your platform, right?  

[GArndt_vacos]

I indeed thought so, but once I started to use the setup productively, the same problems occured, and have been occurring since...

More bughunting for me...

[Vincent]

Sounds for me like multi instance support issue.  

So the heat index was inreased automatically and programmed into the sensor which cause the error message.  

Maybe you can check the globe variables like sensor structure to see if you use them properly in your code.