llama_cpp_dart icon indicating copy to clipboard operation
llama_cpp_dart copied to clipboard

Error loading model: LateInitializationError: Field 'context' has not been initialized.

Open aze1410 opened this issue 6 months ago • 1 comments


  Future<void> _initializeModel() async {
    setState(() {
      _isLoading = true;
      _loadingStatus = 'Requesting permissions...';
    });

    // Request storage permissions
    // await _requestPermissions();

    setState(() {
      _loadingStatus = 'Locating TinyLlama model file...';
    });

    try {
      // Load model from local storage only
      final modelPath = await _loadLocalModelPath();

      setState(() {
        _loadingStatus = 'Loading model into memory...';
      });

      await _setLibraryPath();

      final modelParams = ModelParams();

      final contextParams = ContextParams();
      contextParams.nPredict = 2048;
      contextParams.nCtx = 2048;
      contextParams.nBatch = 512;

      final samplerParams = SamplerParams();
      samplerParams.temp = 0.7;
      samplerParams.topK = 40;
      samplerParams.topP = 0.9;
      samplerParams.penaltyRepeat = 1.1;

      _llama = Llama(modelPath, modelParams, contextParams, samplerParams);

      setState(() {
        _isModelLoaded = true;
        _isLoading = false;
        _loadingStatus = '';
      });

      _addMessage(
        ChatMessage(
          text:
              'Hello! I\'m TinyLlama running on your device. How can I help you today?',
          isUser: false,
        ),
      );
    } catch (e) {
      setState(() {
        _isLoading = false;
        _loadingStatus = 'Error loading model: $e';
        print(_loadingStatus);
      });
    }
  }

  Future<String> _loadLocalModelPath() async {
    final appDir = await getApplicationDocumentsDirectory();
    final modelPath = '${appDir.path}/tinyllama-1.1b-chat-v0.3.Q2_K.gguf';

    final modelFile = File(modelPath);

    if (!await modelFile.exists()) {
      try {
        final byteData = await rootBundle.load(
          'assets/tinyllama-1.1b-chat-v0.3.Q2_K.gguf',
        );
        print('Model loaded from assets: ${byteData.lengthInBytes} bytes');
      } catch (e) {
        print('Failed to load model from assets: $e');
        throw Exception('Asset load failed');
      }
    }

    return modelPath;
  }

  Future<void> _setLibraryPath() async {
    if (Platform.isAndroid) {
      // For Android, the library should be bundled with the app
      Llama.libraryPath = "libllama.so";
    } else if (Platform.isIOS) {
      // For iOS, the library should be bundled with the app
      Llama.libraryPath = "assets/libllama.dylib";
    }
  }

I am facing this error : flutter: Model loaded from assets: 482149856 bytes flutter: Error loading model: LateInitializationError: Field 'context' has not been initialized.

I am running my app in iOS simulator with 3.32.0 flutter version

is this related to libllama.dylib file? because I am running in the iOS similar : I faced this platform exception in : llama.dart 77 line : _lib = llama_cpp(DynamicLibrary.open(libraryPath!));

ArgumentError (Invalid argument(s): Failed to load dynamic library '/Users/azeem/Documents/Projects/on_llm_poc/assets/libllama.dylib': dlopen(/Users/azeem/Documents/Projects/on_llm_poc/assets/libllama.dylib, 0x0001): tried: '/Library/Developer/CoreSimulator/Volumes/iOS_21E213/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 17.4.simruntime/Contents/Resources/RuntimeRoot/Users/azeem/Documents/Projects/on_llm_poc/assets/libllama.dylib' (no such file), '/Users/azeem/Documents/Projects/on_llm_poc/assets/libllama.dylib' (mach-o file (/Users/azeem/Documents/Projects/on_llm_poc/assets/libllama.dylib), but incompatible platform (have 'macOS', need 'iOS-sim')))

aze1410 avatar Jul 12 '25 18:07 aze1410

is most cases this error happen, either the model is unaccessible or corrupt, please double check

netdur avatar Aug 16 '25 18:08 netdur