Kinara ara-2 can't run models

Beginners
1

Hello,
I bought the Kinara ara-2 a few weeks ago and had a tuff time getting the driver running. I got the Drivers and SDK from Geniatech and there were so many errors. I had to recompile the driver many times and finally found a new Makefile on the web. I got the card up and running with a lot of work. Now I can’t run any models because I can’t convert them, the script that does it, llm_model_gen.py is missing and no other scripts work. Does anyone know if Kinara has a new SDK thats actually correct? I really want to get something running on the accelerator and see how fast it runs local LLM’s. Anyone have any ideas?
Thanks!

1 Like
2

idk Kinara but for now…

You are stuck at the exact boundary between “device works” and “device is useful.”

Ara-2 is not a “load Hugging Face weights and run” device. It is a compile-and-run device. You need a model toolchain (converter + quantizer + compiler/scheduler) that produces Ara-2 deployable artifacts. If that toolchain is missing or version-mismatched, you get what you have now: driver loads, card enumerates, but nothing converts so nothing runs. (NXP)

Below is how I’d interpret your case and what I’d do next, in a way that minimizes more dead-end debugging.

1) Background: what “SDK” really means for Ara-2

For Ara-2, “SDK” is not just headers and a runtime library.

NXP’s current description of Ara Software Development Kit (ARA-SDK) explicitly frames it as:

  • an extensible compiler for models including LLMs/VLMs
  • “optimal execution planning” and graph scheduling
  • support for quantization and multiple datatypes (NXP)

The ARA-SDK block diagram also breaks the deployment flow into components like:

  • Model Converter
  • Quantizer
  • Optimizer
  • Scheduler
  • then runtime + drivers on the host side

So: if you got a Geniatech zip that is “drivers + SDK” but does not actually include the converter/compiler toolchain, you will never be able to build runnable LLM artifacts.

2) Why your driver install was painful (and why it matters less now)

Your “recompile driver repeatedly” story is consistent with an out-of-tree kernel driver that’s not packaged cleanly for your kernel version (Makefile/Kbuild mismatch, missing flags, wrong include paths). That is common with accelerator add-in cards.

But the key point is: you already crossed that hurdle. The driver being up mainly proves PCIe plumbing. It does not prove the model toolchain is present.

Right now your blocker is almost certainly toolchain packaging/access/versioning, not Linux kernel work.

3) The missing llm_model_gen.py is a strong signal of an incomplete or mismatched bundle

Given how Ara-2 is positioned, a script like llm_model_gen.py is very likely a wrapper around some combination of:

  • export steps
  • quantization steps
  • compiler invocation
  • packaging into the runtime’s expected format

If it’s missing and “no other scripts work,” the most likely explanations are:

A) You received a runtime-only drop

Resellers often ship:

  • driver + runtime + a couple demos (sometimes the demos assume precompiled models)
    but not the model compiler/converter toolchain.

That lines up with what NXP calls out as part of ARA-SDK (converter/quantizer/scheduler).

B) You have a version mismatch inside the Geniatech package

Docs/scripts reference a filename that moved or was renamed in newer internal builds.
So you see “missing script” but the real issue is “scripts and binaries are from different releases.”

C) The LLM conversion pieces are gated

This happens when the vendor treats LLM enablement as a “supported package” or requires account access, licensing, or an NDA download portal.

This is not hypothetical: an NXP Community thread from late 2025 shows another buyer struggling to get “proper SDK access” to run an example 7B LLM and convert their own model. (NXP Community)

4) Is there a “new SDK that’s actually correct”

The most “official” answer today is: the maintained SDK track is under NXP: ARA-SDK (Ara Software Development Kit).

Evidence:

  • NXP hosts the ARA-SDK product page and describes the compiler/quantization flow. (NXP)
  • NXP’s Ara-2 DNPU page lists Ara Software Development Kit and eIQ Toolkit as software downloads. (NXP Semicondas)
  • NXP’s Kinara acquisition blog says Kinara’s SDK and tools will be integrated into NXP’s eIQ software environment. (NXP)

So yes: there is a “real” SDK path. But in practice it can still be account-gated and messy to obtain via reseller channels. (NXP Community)

5) High-leverage triage: prove whether you have the toolchain in 5 minutes

On your host machine, in the extracted “SDK” directory, check for any of these:

What you should expect in a full toolchain bundle

  • a bin/ directory with large executables
  • directories named like compiler/, converter/, quantizer/, tools/
  • documentation that describes producing a deployable artifact (not just “run this demo”)

What “runtime-only” usually looks like

  • mostly .so libraries
  • sample apps that reference prebuilt assets
  • Python scripts that fail because the underlying compiler binary is not present

Concrete commands (Linux) that don’t assume filenames:

# From the SDK root:
find . -maxdepth 3 -type d -iname "*tool*" -o -iname "*compil*" -o -iname "*quant*" -o -iname "*convert*"

# Look for big executables (compiler toolchains are usually large):
find . -type f -executable -size +5M | head

# Find all references to the missing script name:
grep -RIn "llm_model_gen\.py" . || true

Interpretation

  • If you do not find any serious executables and everything looks like runtime libs and thin scripts, you are missing the model toolchain. That is consistent with ARA-SDK including explicit converter/quantizer/scheduler blocks.

6) Fastest path to “LLM runs and I can benchmark tok/s”

Step 1: stop trying to convert your own model first

Get one known-good runnable LLM demo from the vendor ecosystem.

Why I’m confident demos exist:

  • Kinara publicly demonstrated Qwen1.5-7B at 12 output tokens/sec on a single Ara-2 and explicitly invites people to contact them to see it running. (Business Wires)
  • Geniatech markets Ara-2 modules as supporting transformer/genAI workloads and even names LLaMA 2.0 in AIM-M2 marketing. (Geniatech)

So the shortest benchmark route is: vendor-provided compiled model + vendor runner.

Step 2: get ARA-SDK from the NXP side, not random zips

NXP’s Ara-2 software listings explicitly include ARA-SDK and eIQ Toolkit. (NXP Semicondas)

Step 3: use the vendor-supported host environment

NXP’s eIQ Toolkit downloads show a strong bias toward Ubuntu 20.04 installers (multiple releases) and also provide Windows installers. (NXP Semicondas)
Even if you do not use eIQ itself, this is a practical hint: tooling and dependencies are likely tested most on that baseline.

If you stay on a newer kernel/distro, you may keep paying the “driver Makefile tax.”

7) What to ask Geniatech or NXP for (so you don’t get another broken package)

When you ask, do not say “SDK.” Say:

  1. “I need the Ara-2 model compiler/conversion toolchain (converter + quantizer + compiler/scheduler) that produces deployable artifacts.”
  2. “I also need a known-good LLM demo bundle (compiled model + runner) so I can validate runtime and measure tokens/sec.” (Business Wires)
  3. “Please specify the SDK version/release date and the supported OS/kernel matrix for the driver.”

That wording forces them to send what ARA-SDK is described as containing, not just runtime bits.

8) Why your exact situation is unsurprising right now

Three ecosystem realities are colliding:

  1. Kinara → NXP transition: NXP says Kinara SDK and tools will be integrated into eIQ. That implies moving pieces, renames, and packaging churn. (NXP)
  2. Reseller distribution: Geniatech’s download flow is “we email you a link.” That’s a recipe for partial bundles, outdated zips, and inconsistent versioning. (Geniatech)
  3. Other buyers report access friction: the NXP Community thread shows people stuck trying to get the “proper SDK” to run 7B LLM examples and convert their own models. (NXP Community)

So your experience is consistent with the current state of distribution, not personal failure.

If you want a precise diagnosis

Paste just:

  • the top-level folder names of your SDK directory (one ls output)
  • and the output of find . -type f -executable -size +5M | head

From that, it’s usually obvious whether you have the full model toolchain or a runtime-only package.

Summary

  • Ara-2 requires a converter/quantizer/compiler toolchain, not just drivers.
  • A missing llm_model_gen.py usually means incomplete or mismatched packaging.
  • The most “correct” SDK path is NXP ARA-SDK (and eIQ Toolkit is the broader environment). (NXP Semicondas)
  • Fastest benchmark route: get a vendor-provided compiled LLM demo first, then do custom conversion. (Business Wires)
3

The diagnostic provided is accurate and aligns with the standard architecture of neural processing unit (NPU) deployment flows. The assessment that you are situated at the boundary between “driver enumeration” and “deployment capability” is technically sound.

To ensure scientific rigor and eliminate uncertainty, the following points validate and refine the proposed course of action:

1. Validation of the “Runtime-Only” Hypothesis

The assertion that a missing llm_model_gen.py script indicates an incomplete SDK bundle is consistent with industry-standard SDK packaging for accelerators (e.g., NVIDIA TensorRT, Google Edge TPU, NXP eIQ).

  • Empirical Evidence: A full SDK for a compiler-dependent accelerator (like Ara-2) must contain the toolchain components: Model Converter, Quantizer, and Graph Compiler. If the extracted directory structure contains primarily shared object files (.so) and runtime binaries, but lacks the specific conversion pipelines (Python scripts or compiler binaries), the bundle is defined functionally as a Runtime Environment, not a Development Kit.

  • Conclusion: It is statistically improbable that a “complete” SDK would omit the primary entry point for model generation (llm_model_gen.py) due to user error; its absence strongly suggests version mismatch or incomplete distribution.

2. Technical Analysis of the NXP/Kinara Transition

The transition of the toolchain from a standalone “Kinara SDK” to the integrated “NXP eIQ” environment introduces a packaging discontinuity that explains the current state.

  • Architecture Dependency: The Ara-2 hardware requires a specific graph intermediate representation (IR). If the Geniatech package provides the legacy “Kinara Runtime” but you are attempting to use tools designed for the “NXP eIQ” flow (or vice versa), the binary artifacts will be incompatible.

  • Verification Step: Before debugging drivers, verify the compatibility matrix between the driver version installed (via dmesg or modinfo) and the SDK version. A driver enumeration does not guarantee compatibility with the compiler artifacts produced by a mismatched SDK version.

3. Refined “Artifact-First” Validation Strategy

To strictly validate hardware functionality without the confounding variables of a local toolchain build, you must isolate the Inference Engine from the Compiler Toolchain.

  • Hypothesis: The hardware and driver are functional if and only if a valid, pre-compiled artifact executes successfully.

  • Procedure:

    1. Do not attempt to compile a model locally.

    2. Request a “Golden Reference Image” from the vendor. This is a pre-quantized and pre-compiled model artifact (e.g., a .kbin or proprietary binary format) that is known to pass the internal validation suite of Geniatech or NXP.

    3. Execute this artifact using the provided runtime runner.

  • Expected Outcome:

    • Success: The issue is strictly the absence/misconfiguration of the local toolchain.

    • Failure: The issue lies in the PCIe communication layer, kernel module compatibility, or hardware fault.

4. Recommended Vendor Interaction Protocol

When engaging Geniatech or NXP support, ambiguity in terminology leads to incorrect package distribution. Use the following precise queries to elicit the correct artifacts:

  1. “Provide the ARA-SDK Toolchain Release compatible with Linux Kernel Version [Your Version]. Specifically, I require the Model Converter and Graph Compiler binaries necessary to generate deployable artifacts.”

  2. “Provide a pre-compiled validation artifact (Quantized LLM binary) and the associated runner command line arguments to verify the PCIe link and runtime execution stack.”

Summary
The assessment that you possess a “driver-only” or “runtime-only” installation is highly probable. The path of least resistance for verification is to bypass the local compilation step entirely and execute a vendor-supplied, pre-compiled binary model. This isolates the problem to the toolchain acquisition rather than the hardware configuration.

1 Like
4

Thank you! You both hit the nail on the head! In the readme.md files it seems they provide the toolchain, it has four steps to convert and produce a working LLM, all the other scripts are there except the first one I mentioned. All the software is from Geniatech not Kinara, I opened a ticket with NXP and they said they are waiting on Kinara to provide the SDK etc. I go to the Kinara site to request everything and it points me back to NXP’s site. I’m going to try all your suggestions and see if I can find the correct scripts. This may be a case where I have to wait until Kinara, via NXP, provides us with all the correct software. In the mean time I will try searching for what I need, the three packages I received from Geniatech have a lot in them. Thank you again for all your suggestions, it helps a lot.

Regards,

Mark

1 Like