Skip to content

ERROR: Failed building wheel for tokenizers #3

@RyanFDing

Description

@RyanFDing

Trying to follow the instructions given in section 0 (prerequisites) for running the fine-tuned models in x-fact:
everything works except this line (pip install --editable ./)

I've tried many things, such as installing the rust language and adding vs build tools, but I keep getting this error:


  error[E0557]: feature has been removed
        --> C:\Users\ding\.cargo\registry\src\index.crates.io-6f17d22bba15001f\lock_api-0.3.4\src\lib.rs:91:42
         |
      91 | #![cfg_attr(feature = "nightly", feature(const_fn))]
         |                                          ^^^^^^^^ feature has been removed
         |
         = note: split into finer-grained feature gates

      For more information about this error, try `rustc --explain E0557`.
      error: could not compile `lock_api` (lib) due to 1 previous error

      Caused by:
        process didn't exit successfully: `rustc --crate-name lock_api --edition=2018 C:\Users\ding\.cargo\registry\src\index.crates.io-6f17d22bba15001f\lock_api-0.3.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"nightly\"" --check-cfg cfg(docsrs) --check-cfg "cfg(feature, values(\"nightly\", \"owning_ref\", \"serde\"))" -C metadata=e3b4e41dabc89c13 -C extra-filename=-e3b4e41dabc89c13 --out-dir C:\Users\ding\AppData\Local\Temp\pip-install-d5vow5o_\tokenizers_3feb35558e4a4c8e8d80cedaebf0e349\target\release\deps -L dependency=C:\Users\ding\AppData\Local\Temp\pip-install-d5vow5o_\tokenizers_3feb35558e4a4c8e8d80cedaebf0e349\target\release\deps --extern scopeguard=C:\Users\ding\AppData\Local\Temp\pip-install-d5vow5o_\tokenizers_3feb35558e4a4c8e8d80cedaebf0e349\target\release\deps\libscopeguard-a6a56faef7277818.rmeta --cap-lints allow` (exit code: 1)
      warning: build failed, waiting for other jobs to finish...
      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)

I've tried many fixes and it always gets a little further but still gets stuck on the wheel.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions