Arduino Mega2560 - RX pin lose bytes

[zxia545]

I have config 15 Analog pin to be analog input pin and then I found sometime Arduino did not received correct message that I sent from Python. (I have verified the message send from python is correct)
It seems like a problem with continue reporting back analog pins and digital pins from TX back to python. The RX sometime will lose few bytes.

So, I used oscilloscope to monitor RX pin from Arduino, and sending [0x91, 0x40, 0x00] and [0x91, 0x00, 0x00] each 0.5 second. But sometimes it will drop the last or last two bytes.

This problem can be solved by changing the baud rate from 115200 to 9600, but then for those pins be config as analog input become a problem since the baud rate is dropping. Another way to hide this issue is only setting one or two pins to be analog input pin.

I also try to use an external USB to TTL converter to bypass this issue, it kind of fix the RX issue but the data sending back from Arduino to python is not correct at using the original USB to TTL converter micro in Arduino.

[MrYsLab]

Thanks for reporting tthis. I don't have the resources to run that number of analog inputs, but this suggestion may solve the issue. You can change the analog input scan rate by calling set_analog_scan_interval The default scan rate is 19 ms.

Please let me know if this solves the issue.

[zxia545]

I just did with the sampling interval set to 50ms. But sadly, this problem still exists :(

[zxia545]

Thanks a lot, but I am currently using the pymata4 with firmata-express :(.
So can I call set_sampling_interval in pymata4 to achieve the same result? And what if disable all the analog input pins polling that Arduino only sends analogRead value when receiving an inquiry for an analog pin (a command send from pymata4)?

Also wondering why you decided to use the polling mechanism for reading analog input and digital input. Is it because you want to increase the speed of the python reading value of analog and digital pins?

[MrYsLab]

Hi Tony,
I have moved this to a discussion since it is about pymata4 which is now archived. You can try changing the sampling interval, but the code may still overwhelm the serial buffer of the Mega2560. Unfortunately, Arduino does not provide any serial flow control.
One solution is to add a small delay after this line in FirmataExpress.ino.
In FirmataExpress, the way analog disable works is that the pin is reset to a digital input. If your analog devices never cross the digital crossover threshold, all will work, but if not, you will get digital input messages when using callbacks. Telemetrix, on the other hand, actually suppresses sending messages when reporting is disabled.
The polling mechanism is part of the original Firmata code which is the basis of FirmataExpress. I am not sure what the alternative to polling would be, so I am probably just misunderstanding your point.

[zxia545]

Sorry, I was thinking about changing the code of Firmata to remove the part keep sending current analog read value back to python site and create a callback function to handle analog read request such that Firmata only read analog pin value and send it back to python when received a request message (Also need to add a request command and method from python site).

[MrYsLab]

If having Python poll analog and digital pins makes more sense for your application, then you should implement it. Firmata was developed to have the Arduino automatically monitor data changes and report them back via the serial interface. Because one can configure the Arduino to have a variable number of pins enabled for input and asynchronously have the Arduino receive commands, such as setting a digital output state, it is not real-time in any sense of the word.
Firmata-express makes real-time behavior even less predictable because it adds handling for devices like the HC-SR04 and DHT sensors.
Firmata also uses the concept of a 7-bit byte which adds overhead. It also uses both fixed-length and variable-length messages in its reports, adding an unnecessary level of complexity.

[MrYsLab]

I just realized your post says the RX pin. What commands are you sending from Python to the Arduino when the issue occurs? Have you tried adding a small delay between sends?

[zxia545]

Yap, so my test was sending a message from python every 0.2 seconds. This problem come up after config more than two pin to be analog input pins. Please check the script I used for testing.

[zxia545]

Also, the RX bytes missing issue can be solve by using an external USB to TTL converter. But another issue come up when using external one, which is return value of analog_read and digital_read methods from python site is not correct. I guess the TX from Arduino to USB have another problem, but I did not research more into it.

[zxia545]

I think I may find the answer. Previously I run pymata4 on Raspberry PI and I did another test which run the same pymata4 script on Window device. The Arduino RX sometime still having some bytes delay, but it happened less frequent.

In my point of view, this may because of Raspberry PI and Window device they are not Real Time OS. But PI compared with Window it runs a little slower such that the problem occurring much more.

[zxia545]

I take a screenshot of pin 14 when my testing script is running on PI.

But when I use Window device to run it, pin 14 never behave like this.

Test Script:

def testPymata4():
port=find_port_by_id()
board = pymata4.Pymata4(com_port=port[0], baud_rate=115200)
# board = pymata4.Pymata4(com_port="COM14")

# set interval to 50ms
# board.set_sampling_interval(50)


board.set_pin_mode_digital_output(14)  # use pin number not name
board.set_pin_mode_digital_input(22)

board.set_pin_mode_tone(47)


board.set_pin_mode_analog_input(0)
board.set_pin_mode_analog_input(1)
board.set_pin_mode_analog_input(2)
board.set_pin_mode_analog_input(3)
board.set_pin_mode_analog_input(4)
board.set_pin_mode_analog_input(5)
board.set_pin_mode_analog_input(6)
board.set_pin_mode_analog_input(7)
board.set_pin_mode_analog_input(8)
board.set_pin_mode_analog_input(9)
board.set_pin_mode_analog_input(10)
board.set_pin_mode_analog_input(11)
board.set_pin_mode_analog_input(12)
board.set_pin_mode_analog_input(13)

startTime = time.time()
while True:
    if time.time() > startTime + 90:
        break
    print("ON")
    board.digital_write(14, 1)  # just use pin number and integer value

    time.sleep(0.1)
    print("OFF")
    board.digital_write(14, 0)  # just use pin number and integer value

    time.sleep(0.1)

[zxia545]

But I still have a question about firmata-express. Since once it received the header, it will just wait for number of data bytes and response to it https://github.com/MrYsLab/FirmataExpress/blob/7149014763f3e0395ca9f718c4b523c56c2065e1/FirmataParser.cpp#L159

But from the screenshot, Arduino did not change PIN 14 value by more than 0.5 second. Keen of interest to know if you can explain it a little bit :)

[MrYsLab]

I am unsure of what you mean by Arduino did not change PIN 14 value will more than 0.5 second.

Do you get the expected results if you only enable pin 14 as a digital output and do not enable the analog input pins? If the answer is yes it is because Firmata constantly checks for input in the loop function. It also continuously checks for digital input changes.
When analog inputs are enabled, then the sampling interval variable is checked. That interval has a default of 19ms. Depending upon how many analog inputs are enabled and if other devices, such as an HC-SR04, are added to the mix, the frequency of checking for new incoming messages and checking for digital input changes varies. The Arduino then becomes less predictable in responding to incoming messages.

Please let me know if this answers the question.

[zxia545]

Oh, I see it seems like Arduino just less predictable in response, as you said. Maybe for my problem, the best solution is to remove the Arduino report mechanism and let python send requests to Arduino for querying analog or digital read on different pins.

[MrYsLab]

If you run into any issues when trying this out, if I am able, I would be happy to answer any questions you might have.

[zxia545]

Thanks a lot.

[zxia545]

If you run into any issues when trying this out, if I am able, I would be happy to answer any questions you might have.

I created two new callback function for handle analog and digital read request sending from Pymata4 and disable the polling for analog and digital pins. It seems working now. Thanks for your help!

[MrYsLab]

I am glad that it is working out for you.