MicroPython 'memoryview' Object

Contents

What is the memoryview Object?

The memoryview object allows MicroPython code to access the internal data of another object that supports the buffer protocol without firstly copying the object.

Object types that support the buffer protocol are byes, bytearray, string and uarray.array.

Whenever some action is performed on an object (e.g. using the slice operator on an array), MicroPython needs to create a copy of the data.

If there is a large quantity of data to work with (eg. binary data of an image), MicroPython would be unnecessarily creating copies of huge chunks of data. In the worst case this might cause the memory limited microcontroller to reach the limit of its memory.

Using the buffer protocol and a memoryview object, another object can be given access to use or modify the large data without copying it. It just uses pointers to access the original data source. This makes the program use less memory and increases the program's execution speed.

Creating a memoryview Object

A memoryview object is obtained with the memoryview() function.


Syntax
memoryview(obj)

Where:
  obj is an object whose internal data is to
  be exposed. It must support the buffer
  protocol i.e. be one of bytes, bytearray,
  str or uarray.array types.

Example:
b = bytearray('ABCDE')
m = memoryview(b)

print(b[0]) ⇒ 65
print(m[0]) ⇒ 65
print(chr(m[0])) ⇒ A

print(m) ⇒ <memoryview>
print(m[1:3]) ⇒ <memoryview>
print(m[1:3][0]) ⇒ 66
print(list(m[1:3])) ⇒ [66, 67]
print(bytes(m[1:3])) ⇒ b'BC'
print(bytes(m[1:3]).decode()) ⇒ BC

The above example empasises that a memoryview object is merely a pointer to an object that supports the buffer protocol. Using an index e.g. m[0] returns an 8-bit byte at the referenced position within the buffer.

On the other hand, using the slice operator e.g. (m[1:3]) on a memoryview object returns another memoryview object that is a pointer to the sliced section of the buffer data. Each individual 8-bit byte of the slice pointed to by the new memoryview object can be accessed by index e.g. m[1:3][0].

Using memoryview Objects

If the buffer data is updatable i.e. the buffer data type is mutable (bytearray, uarray.array), then elements can be updated through the memoryview. This updating occurs directly on the underlying data without copies being made. This is very efficient especially if the underlying binary dataset is very large.

Example 1 demonstrates just how easy it is to perform update operations on a large binary dataset using a memoryview object.

Example 1 

# Demonstrate the use of a memoryview object
# to update a buffer protocol object.
# The buffer protocol object type must be
# mutable to successfully change a value.

# Declare a buffer protocol object.
b = bytearray([0, 1, 2, 3, 4, 5])
print('bytearray example')
print(b)

# Declare a memoryview that references
# the bytearray.
m = memoryview(b)
for i in range(len(b)):
    print(m[i], end=' ')

# Update the value of the second
# byte in the bytearray using
# the memoryview object.
m[1] = 6
print('\n\nUpdating the second byte...')
print('Updated bytearray:')
print(b)

# Attempt to update an immutable buffer
# protocol type will cause an error.

# Declare a string
s = 'Hello World'
print('\nstring example')
print(s)
# Declare a memoryview object
m = memoryview(s)
for i in range(len(b)):
    print(m[i], end=' ')
# Attempt to change a value in
# the string through the memoryview
# object. This will cause an error.
print('\n\nAttempting to change a value...')
m[0] = 6
Output: 

bytearray example
bytearray(b'\x00\x01\x02\x03\x04\x05')
0 1 2 3 4 5 

Updating the second byte...
Updated bytearray:
bytearray(b'\x00\x06\x02\x03\x04\x05')

string example
Hello World
72 101 108 108 111 32 

Attempting to change a value...
Traceback (most recent call last):
  File "main.py", line 40, in <module>
TypeError: 'memoryview' object doesn't support item assignment

As Example 1 shows, attempting to change a value of an immutable buffer protocol object type (e.g. string) will cause an error regardless of whether an memoryview object is used or not.

Example 2 demonstrates how memory efficient it can be to use a memoryview object when performing large slice operations. In this example a large slice operation on a huge (near RAM capacity) binary dataset without using a memoryview object will cause a fatal out of memory error.

Example 2 

# Demonstrate the use of a memoryview object
# to perform a slice on a buffer protocol
# object containing a large binary dataset.

# Using a memoryview object means that
# the buffer won't be copied when the
# slice operator is used.

import urandom # To generate random numbers

# Define the size of the buffer in bytes
# such that it almost exceeds the mico:bit's
# total RAM.
size = 60000
# Declare the buffer and fill with null data.
buffer = bytearray(size)

# Update the entire buffer with random data.
for i in range(size):
    buffer[i] = urandom.randint(0, 255)
 
# Declare a memoryview object
# pointing to the buffer.
m = memoryview(buffer)

# Take a large slice of the buffer
# using the memoryview object.
# This will work fine as the
# buffer won't be copied.
# This will return another memoryview object
# that references the slice.
slice1 = m[0:size-1000]
print('Take huge slice using memoryview object')
print('First 3 bytes of the slice:')
print(slice1[0], slice1[1], slice1[2])

# Take a very small slice of the buffer
# without using a memoryview object.
# The slice will be a copy of the data
# from the buffer that will just
# fit in the micro:bit's RAM.
slice2 = buffer[0:5]
print('\nTake slice without memoryview object')
print('First 3 bytes of the slice:')
print(slice1[0], slice1[1], slice1[2])

# Now take a very large slice without
# using a memoryview object. An attempt
# will be made to make a copy of the
# buffer data. The micro:bit will
# not have sufficient RAM so an exception
# will be raised.
print('\nAttempting huge slice')
print(' without using memoryview object')
slice3 = buffer[0:size-1000]
Output: 

Take huge slice using memoryview object
First 3 bytes of the slice:
72 10 39

Take slice without memoryview object
First 3 bytes of the slice:
72 10 39

Attempting huge slice
 without using memoryview object
Traceback (most recent call last):
  File "main.py", line 56, in <module>
MemoryError: memory allocation failed, allocating 59000 bytes

Most MicroPython programmers will never use memoryview objects but they are still useful to to be aware of… just in case.

<  Previous