2

I recall being able to read in the display buffer of NXT 2.0 bricks over Bluetooth/USB by issuing Read IO Map system commands. I've looked through the bytecodes and communication documentation for the EV3, but it's not clear how to do this with the EV3. Does anyone have any pointers?

3

I think you should be able to do this. The screen is represented by a framebuffer device. In Linux, everything is a file, so even the framebuffer is a file, namely /dev/fb0.

So, you should be able to read the display by reading the /dev/fb0 file using something like this:

EDIT: Using direct commands didn't work out so well, so here is a sample using system commands.

Getting file content
--------------------

Used to upload datalog files - file handle is only closed when reaching EOF
and file is not open for writing

BEGIN_GETFILE:

  Bytes send to the brick:

  xxxxxxxx0196xxxxxxx
  bbbbmmmmttssllllnnn...

  bbbb = Bytes in massage, mmmm = message counter, tt = type of command,
  ss = system command, llll = max bytes to read, nnnn.... = path


  Bytes send to the PC:

  xxxxxxxx039600xxxxxxxx00xxx
  bbbbmmmmttssrrllllllllhhppp...

  bbbb = bytes ion massage, mmmm = message counter, tt = type of command,
  ss = system command, rr = return status, llllllll = File size, hh = Handle,
  ppp... = payload


CONTINUE_GETFILE:

  Bytes send to the brick:

  0700xxxx019700xxxx
  bbbbmmmmttsshhllll

  bbbb = bytes in massage, mmmm = message counter, tt = type of command,
  ss = system command, hh = handle, llll = max bytes to read


  Bytes send to the PC:

  xxxxxxxx039700xxxxxxxx00xxx
  bbbbmmmmttssrrllllllllhhppp...

  bbbb = bytes in message, mmmm = message counter, tt = type of command,
  ss = system command, rr = return status, llllllll = File size,
  hh = Handle, ppp... = payload

Opcodes come from here System commands come from here. The size returned may be 0 since this is not a real file, but the framebuffer size is 7680 bytes (comes from here, i.e. (178+2)/3*128). Also, the maximum number of bytes allowable for max bytes to read depends on the connection type (USB, Bluetooth or Wi-Fi). 1000 should be a safe number.

Now, hopefully you have the data, but you still need to decode it. The framebuffer has an unusual format where there are 3 pixels per byte (see datasheet).

Assuming you have packed all of the data read into a single 7680 byte array you can use nested for loops to convert the data to whatever image format you like. For simplicity, I am going to use a .pbm. Just doing this in pseudo code...

// framebuffer data - 60 bytes/row * 128 rows
buffer[60*128]

// write the PBM header
write_line("P1")
write_line("178 128")
for (row = 0; row < 128; row++) {
    // last byte in each row only has 1 pixel, so only loop first 59 bytes
    for (offset = 0; offset < 59; offset++) {
        var b = buffer[row * 60 + offset];
        // 3 pixels per byte

        // bit 7 (or 6) is pixel 0
        write((b & 128) ? "1 " : "0 ")
        // bit 4 (or 3) is pixel 1
        write((b & 16) ? "1 " : "0 ")
        // bit 1 (or 0) is pixel 2
        write((b & 2) ? "1 " : "0 ")
    }
    // the last pixel
    b = buffer[row * 60 + offset]
    write_line((b & 128) ? "1" : "0")
}
  • I tried getting OPEN_READ/READ_BYTES working with an actual file first. OPEN_READ returns the correct file size however the handle is always 1 (0x01 0x0 0x0 0x0 in little endian) no matter how many times I make the call. If I subsequently make a call to GET_HANDLE to get the handle for the same file path, I get (0xFF 0xFF 0x0 0x0) for the file handle (instead of 1). Both commands succeed. When I try to actually read the file with READ_BYTES (trying both handles: 0x1 and 0xFFFF), it fails. Something is not right. – Matt Rajca Dec 25 '15 at 6:41
  • Look's like I made at least one mistake, Handle is 16-bit. Here is a sample that may help: github.com/mindboards/ev3sources/blob/… – David Lechner Dec 25 '15 at 18:40
  • Looking at the source code, it looks like if a file fails to open, the command will still return successfully, although I would expect the handle to be 0 and not 1 as you observed. Source: github.com/mindboards/ev3sources/blob/… – David Lechner Dec 25 '15 at 18:53
  • I was able to get things to work by sending the OPEN_READ/READ_BYTES commands at once with a "compound command" (as the docs call it on page 26 of the EV3 Communication Developer Kit PDF), rather than individually. This even works with /dev/fb0, however I cannot read more than 1024 bytes now since the global variable space is 10-bit. If there was an opcode for seeking to a different position (there is not), we could go through the OPEN_READ/READ_BYTES cycle again, this time starting at a different offset. – Matt Rajca Dec 25 '15 at 23:03
  • Could you save the file contents to an array variable and then read the the array 1000 or 1024 bytes at a time using opARRAY(READ_CONTENT, ...)? – David Lechner Dec 26 '15 at 2:15

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.