Niepce November 2022 updates

Here are the updates in Niepce development for November 2022.

I'll try to bring these updates on a more regular basis. In case you missed it, there was a May update.

No more cbindgen

For this one, from a user standpoint, there is nothing really visible beside regressions, but given there won't be a release for a while, it's a non issue. Things will be addressed as they are found.

I decided to bite the bullet and get rid of cbindgen. A few reasons:

• The way I implemented it caused build time to be much longer unless I disabled the bindings generation. Turn out that cbindgen parses the whole code based a second time to automatically generated the bindings, and I did make this step opt-out instead of opt-in to make sure the code was in sync. There would probably have been a different way to go about it, but I didn't.
• cbindgen is one way (calling Rust from C/C++) and this limited its purpose. I had at one point bidirectional bindings with bindgen but it was also fragile. Boath are great tools, but like any tool they have limitations. I would still use cbindgen to generate the C headers of an API I want to export, and bindgen to generate the Rust FFI (Foreign Function Interface) for Rust bindings to a C API.
• It's still a C API with all the issues. Raw pointers and manual cleanup, this made things more fragile at the FFI level.

I transitionned to cxx. cxx is a fantastic tool for bi-directional safe bindings between C++ and Rust. It provide strongly typed bindings, support for Box<> and C++ smart pointers, and C++ class syntax.

It wasn't an easy transition though, and I had already tried in the past. But really it was worth it.

Using cxx

The gist of cxx is that you write a ffi section that will declare, using Rust syntax, an interface for your code at the edge of Rust and C++. And then cxx will generate the glue code to call a C++ method from Rust or have C++ code call Rust methods. As a bonus, it will allow direct use of strings, vector, Box, std::shared_ptr, std::unique_ptr, which mean that you can have reasonable expectations of resource management and standard types that cbindgen required you to deal with. This is truly a game changer in comparison. A lot of the convoluted FFI I had to write before is not handled by the tooling. And the code is more idiomatic.

use crate::mod1::Type1;

#[cxx::bridge(namespace = "ffi")]
mod ffi {
    extern "Rust" {
        type Type1;

        #[cxx_name = "Type1_create"]
        fn create() -> Box<Type1>;
        fn method1(&self, name: &str) -> i32
    }
}

In C++ to use this you'd write:

#include "the_bindings.hpp"

void f() {
    ::rust::Box<ffi::Type1> object = Type1_create();
    uint32_t value = object->method1("some_name");
}

Note that the_bindings.hpp here is a generated header, so is its counterpart the_bindings.cpp (you pick the name) that should be compiled with the rest of your code.

However, there are pitfalls. Niepce is split is three crates. This is a problem as cxx doesn't handle cross-crate bindings as easily as it could.

For example, imagine there is a type A in crate_one, that likely have a binding. crate_two has a type B where one of the method needs to return a type of crate_one::A. You can't easily. One of the trick is to declare A as a C++ type which requires implementing the trait cxx::ExternType. But if you don't control the crate, then it won't work as you can't implement a trait outside of either the crate that declares it or that declare the type you are implementing the trait for.

This led to the other problem: the code uses gtk-rs and the bindings needs to deal with gtk types. The solution I found is to declare extern C++ types that are name like the gtk type and in the Rust code to cast the pointer to match the underlying _sys types. For example a *mut ffi::GtkWidget as *mut gtk4_sys::GtkWidget. The compiler can't help validating the type, but at least it's localized in the bindings, and it's not worse than before.

Moving forward

And in parallel, as this was the goal, I ported large chunk of the UI code to Rust. Removing C++ code is so satisfying.

So moving forward, this lay the ground work to move more code to Rust as to not write new C++ code.

Lightroom Import

This was a big feature branch I finally merged almost a year later. With that previous change the merge was bit paintful. In various order, this provide a few new features that are not specific to the importer:

• catalog update (when the database format changes)
• albums, with a limited feature set
• import of Lightroom™ 4 and 6 catalogs
• provide some infrastructure to import others (I have some plans)

This is currently very experimental as there are possibly a lot of uncovered issues, but having it in the main branch is really the first step.

Album support

With Lightroom™ import come the need to support albums, ie collection of images. So now you can create and delete albums, and the Ligthroom™ importer will import them.

The main user interaction for album is drag and drop. Drag an image onto an album to add it. Drag and drop isn't implemented. Also renaming them isn't implemented either.

Preparing for drag and drop support

Before implementing drag and drop between list widgets (the grid view or thumbnail strip and the workspace), we'd better stop using the now deprecated widgets. This mean it is time to remove GtkIconView and GktTreeView as I would hate having to implement drag and drop twice.

Now, the grid view / thumbnail strip no longer uses GtkIconView. There are still a few visual adjustments to perform, notably how to get square cells, or rethink the layout. This also got rid of the last leftovers from cbindgen.

The workspace (the treeview that list the content of the catalog) is a bit more tricky as the API for tree view with GtkListView has a few limitations and complexities, but it's mostly done as to have the functionality. It's the last big patch in November.

On the left, before the change. On the right, after the change.

These expander triangles are what needs fixing and are caused by limitation in Gtk4: Gtk.TreeListRow:expandable is immutable, and the expander is always visible if the property is true.

Adwaita

The single most visible change is the addition to libadwaita support that by itself change the way the UI look by adding just one line of code.

Before:

After:

It landed before the workspace change. The only other code change needed was the handling for the dark/light them preference.

Current short term plans

There is a long list of stuff I want to do for an initial release, which is a decade overdue ;-), and which is unlikely to happen soon.

Upcoming things are, beside finishing album support, porting the build system to meson (patch in progress), finish the file import (and port it to Rust), finish the library import with possible more formats, with actual performance improvements as import is still very slow. Notably the USB Mass Storage use libgphoto2 instead of copying directly, and library import hitting the sqlite database quite hard.

There is also the raw processing question. Currently I use libopenraw for thumbnailing, and GEGL built-in processing for rendering (it uses LibRaw). Nothing that I consider functional, just the plain minimum to view an image. I have a few ideas involving "importing" a lot of other third-party code (C/C++), but I want to experiment first with it, as there is a large network of dependencies.

Thank you for reading.