Skip to content

Lua fixes part 2

Alex requested to merge branch/lua_fix into branch/unified

Summary:

  • Backend:

    • simple (unloadable) scripts in addition to previous concept of "complex" scripts (dmilligan)
    • simple scripts will 'unload' when finished running, freeing memory
    • umm_malloc (custom memory allocator for Lua - much faster)
    • small memory backend tweaks and fixes to behave a little better with Lua
    • print last error in menu, tag errors with [SCRIPT.LUA]
    • scripts can be edited in EDITOR.LUA from context menu
    • autorun can be turned on/off by user for each script
    • each script can have "command-line" arguments (currently used with editor.lua)
    • strict.lua only loaded once (faster)
    • doc updates
    • bunch of other minor fixes
  • global:

    • removed global shoot() (use camera.shoot instead)
  • camera:

    • camera.shoot(should_af) now calls take_a_pic (syntax changed, removed the wait parameter)
    • camera.bulb(duration_seconds) fixed (before, docs said seconds but the argument was ms)
    • camera.burst(num_pics) - take N pics in burst mode
    • backend: fix waiting after taking a picture
  • battery:

    • battery.time_remaining (estimated)
  • display:

    • display.print: allow transparent text on transparent background (discussed here; changes done in ML core)
    • display.print: rough tab support (4 spaces) for editor.lua (fixed in ML core)
    • display.screenshot: fix color palette outside ML menu (fixed in ML core)
  • dryos:

    • dryos.rename(src, dst)
  • interval:

    • fix data type of interval.running (bool)
  • tasks:

    • task.yield(): workaround to prevent unsafe usage (one interruption is fine, going back and forth between tasks is not)
  • key:

    • key.wait() uses the task.yield() workaround
    • key.press(KEY.HALFSHUTTER) now works (this might affect the behavior of many other core functions that call SW1/SW2, to be tested)
    • fast scrollwheel events on DIGIC 5 models fixed
    • fix handling unknown events
    • use emulated Q button for cameras without it (so modules and scripts can simply test for the Q event)
  • lens:

  • lv:

    • fix lv.start() in various GUI modes and edge cases (fixed in ML core)
  • menu:

    • removed run_in_separate_task
    • menu.set: backend fixes + API tests (changing menu values by string works much better)
    • menu.get: option to return string; returns nil if menu not found
    • menu.set: returns nil if menu not found
    • menu.select(parent, entry): select ML menu item by name
    • removed menu customization fields (these are internal to menu backend)
    • menu.hidden now handles programmatic menu hiding (todo: same for core code)
  • movie:

    • raise error if ML menu is open (as it works by pressing REC/LV button, which is used in menu)
    • backend checks for duplicate menu entries
  • scripts:

    • api_test, calc, editor, hello, pong, sokoban are now "simple" (unloadable) scripts
    • pong: minor fix (ML menu was not always fully erased before starting the game)
    • recdelay: minor fixes (reasonable defaults; close ML menu)
    • editor: minor fixes (stop on closing ML menu; allow command-line arguments)
    • api_test.lua: new tests (keys, picture taking, movie recording, autofocus, multitasking)
    • unload.lua: test for unloading simple scripts (even if they create tasks and register event handlers, they can still be unloaded if certain conditions are met)

Some changes unrelated to Lua:

  • fine-tuned memory backend behavior (found a case where it was running out of memory)
  • selftest module: stub tests for autofocus functionality
  • max 64 modules (from 32)
  • modules made of multiple source files can now include "module.h" without much trouble
  • bumped API version

This PR also includes dynamic-my-menu, because of its tight integration with Lua menu API changes.


Regarding memory:

Looks like what Lua is doing to the memory backend is pure craziness...

Loading all scripts, including those in the extra directory, and rawbench.lua, allocates about 7000 tiny blocks. Memcheck overhead was 55% before these changes, 27% after.

Fragmentation becomes a big issue - I managed to have about 1MB free in AllocateMemory, but the largest block was smaller than 2K. This can easily cause Canon tasks to fail with ERR70.

I think we should implement a way to load the scripts on demand, rather than autostarting all of them.

I'm also thinking to use an allocator optimized for lots of tiny blocks, such as umm_malloc. Would be nice to start with a contiguous block for this allocator (say 256K) and grow it as needed (maybe by adding one more such block when needed). This way, I hope to minimize fragmentation in the main memory pools, and I also hope to speedup the thousands of realloc calls Lua is making.

Anyone has experience with umm_malloc or other similar library?

Other reports:

Merge request reports