Clojure is Capable

Sure, I will. In src/ruse/core.clj you can find the custom implementation.

Set up the name space, its a bit larger than last time:

(ns ruse.core (:require [clj-http.client :as client] [ruse.util :as u] [ :as dog] ) (:import (jnr.ffi Platform Pointer) (jnr.ffi.types off_t size_t) (ru.serce.jnrfuse ErrorCodes FuseFillDir FuseStubFS) (ru.serce.jnrfuse.struct FileStat FuseFileInfo) (ru.serce.jnrfuse.examples HelloFuse) ( File) (java.nio.file Paths) (java.nio ByteBuffer) (java.util Objects)) (:gen-class))

Oh yea, and if you're so inclined, open up src/ruse/util.clj to see what that's all about. Most of it is just standard utility calls (very specific string parsing etc.), but one thing that's pretty interesting is this macro:

(defmacro lexical-ctx-map "Pull in all the lexical bindings into a map for passing somewhere else." [] (let [symbols (keys &env)] (zipmap (map (fn [sym] `(quote ~(keyword sym))) symbols) symbols)))

This essentially lets us "bubble up" all lexical variables into a map that we can pass to something else.

See this sample:

(let [x 1] (let [y 2] (lexical-ctx-map)))
;; Evals to: {:x 1 :y 2}

So, if you had a look at the JNR library, or have used any Java interop in the past, you should be aware that to extend a class, you just make a proxy call and give it your method overrides. The JNR FuseStubFS class closely resembles the functions you would define for raw libfuse integration in a C file:

(defn fuse-custom-mount [] (proxy [FuseStubFS] [] (getattr [path stat] ; string , jni (cond (u/member path stub-dirs) (getattr-directory (u/lexical-ctx-map)) (dog/dog-exists? path) (getattr-file (u/lexical-ctx-map)) :else (enoent-error))) (readdir [path buf filt offset fi] ;; Here we choose what to list. (prn "In readdir") (if (not (u/member path stub-dirs)) (enoent-error) (readdir-list-files (u/lexical-ctx-map)))) (open [path fi] ;; Here we handle errors on opening (prn "In open: " path fi) (if (and (u/member path stub-dirs) (not (dog/dog-exists? path))) (enoent-error) 0)) (read [path buf size offset fi] ;; Here we read the contents (prn "In read" path) (if (not (dog/dog-exists? path)) (enoent-error) (read-fuse-file (u/lexical-ctx-map))))))

Here you can see what's happening - we're mapping out / overriding 4 methods:

  • getattr: what your OS gives commands like `ls` when they want

attribute info,

  • readdir: the list of files/directories that are shown on tools like `ls` and file managers
  • open: when something goes to 'open' a file, if it should error out or not.
  • read: the code/logic for actually serving the contents to the user (remember that byte-array?)

Also take note of the code organization - unlike a Java class where all those calls have to be defined inline, resulting in a massive class definition, in Clojure we can call plain old top level functions from within these method overrides.

See the macro calls to lexical-ctx-map? That's so our tightly coupled functions that handle the fuse functionality don't need to redundantly bind / pass around map data or scalar arguments from the methods to the functions etc. If we had kept the code inline, it would have had the same variables available, so this is pretty safe with the tight coupling.

Speaking of those defns, lets check them out one at a time, starting with our getattr implementations (one for directories, one for files):

(defn getattr-directory [{:keys [path stat]}] (doto stat (-> .-st_mode (.set (bit-or FileStat/S_IFDIR (read-string "0755")))) (-> .-st_nlink (.set 2)))) (defn getattr-file [{:keys [path stat]}] (doto stat (-> .-st_mode (.set (bit-or FileStat/S_IFREG (read-string "0444")))) (-> .-st_nlink (.set 1)) ;; Fake size reporting - 10MB is plenty. (-> .-st_size (.set (* 1024 1024 1)))))

So, if you've ever seen GNU/Linux (or I guess any UNIX/POSIX file system permissions before) you'll likely recognize the octal permission masks.

The only thing of real interest/note here is that we're giving a fake size value to our JNR struct. That's fine, as long as the number here is higher than the real bytes any given file handle will contain (I chose something reasonable like 10MB instead of 10TB in case a tool out there attempts to always read the full bytes from a getattr call vs stopping at the EOF signal - locking it up like that could be bad).

If you were making a really fancy REST API integration, maybe you would query all records on each `ls` invocation to serve real file sizes. We're not that fancy.

Lets go onto how to provide the list of directories/files:

(defn readdir-list-files-base "FILES is a string col." [{:keys [path buf filt offset fi]} dirs files] (doto filt (.apply buf "." nil 0) (.apply buf ".." nil 0)) (doseq [dir dirs] (.apply filt buf dir nil 0)) (doseq [file files] (.apply filt buf file nil 0)) filt) (defn readdir-list-files [{:keys [path buf filt offset fi] :as m}] (cond (= "/" path) (readdir-list-files-base m (dog/get-breeds) []) ;; Pop off leading slash and show the list of breeds. :else (readdir-list-files-base m [] (dog/get-dog-list! (subs path 1))) ))

So, we have a "base" function, which handles the full map of method params, as well as a string vector of dirs and files. For any given call that isn't in the root directory (the `/` is your FUSE mount root, not your OS level root) it will check the path, so `/whippet` will become `whippet`, at which point we will query our dog API to get a list of all whippet pictures.

Had we queried against the root directory, we would instead give a list of all the dog breeds as our directories.

So, how do we give these wonderful images to the user?

(defn read-fuse-file [{:keys [path buf size offset fi]}] (let [ bytes (dog/get-dog-pic path) length (count bytes) bytes-to-read (min (- length offset) size) contents (ByteBuffer/wrap bytes) bytes-read (byte-array bytes-to-read) ] (doto contents (.position offset) (.get bytes-read 0 bytes-to-read)) (-> buf (.put 0 bytes-read 0 bytes-to-read)) (.position contents 0) bytes-to-read))

In this case, we simply use our byte-array returning API call, and chunk it out in a ByteBuffer wrapper to the user (most software that reads files does it in chunks, giving the OS the offset + size of data they want to read, so just plopping it all out at once will not work well in 99% of your use cases here).

Oh, and in that main proxy definition, we also had relied on some "stub-dirs" variable. That was me being very lazy and just pre-querying to override some directories I had hard coded during prototyping:

(defn set-stub-dirs [] (->> (conj (map #(str "/" %) (dog/get-breeds)) "/") (into []))) (def stub-dirs (set-stub-dirs))