Arm port

[XapaJIaMnu]

Description

RFC on the initial ARM port work
List of changes:

• Added ARM cmake definition
• Added simd_utils to translate x86_64 simd intrinsics to ARM
• Added RUY for gemm implementation

Added dependencies: Ruy, simd_utils

How to test

Download a model and run it:

nik@nikserver2:~/enes$ ~/marian-dev/build/marian-decoder -m model1.npz --mini-batch 1 --maxi-batch 1 -v vocab.esen.spm vocab.esen.spm --quiet --quiet-translation
Hi there, I am running on an ARM PC.
Hola, estoy corriendo en un PC ARM.

TODO

• Use RUY to make int8 inference on ARM
• Check how to make it work on ARMv7 (Currently hardcoded to v8, might need to use a different simd-arm library)
• Maybe get rid of OpenBlas? Ruy doesn't have full sgemm support but we fake it. We need Ruy for doing 8bits and FAISS requires a lot of BLAS routines (hence openblas)
• More proper guards?
• Currently we are not vectorising as much as we can because we are only using ARM SIMD for the SSE codepath, but we could technically do AVX. Also proper ARM intrinsics might be faster.

Checklist

• I have tested the code manually
• I have run regression tests
• I have read and followed CONTRIBUTING.md
• I have updated CHANGELOG.md

[hieuhoang]

Looks good. Checked that it still works with some of my existing models on (Intel) CPU and GPUs.

[XapaJIaMnu]

I am not particularly happy about this.

[hieuhoang]

Rome wasn't built in a day

[XapaJIaMnu]

Do you have an M1 to test on ? We should test RUY vs OpenBLas vs AppleAccelerate to determine what sgemm implementation to use here.

[hieuhoang]

On the case with M1

[jerinphilip]

Ruy path was intended for Android. The usual BLAS was painful to compile. Since not available via package manager or anything we'd have to go compile-route with a submodule addition if going for OpenBLAS.

[hieuhoang]

Did you guys ever get the int8 stuff to work? Did you use RUY?

[XapaJIaMnu]

I haven't ported it from our fork yet. Int8 with ruy works in our port.

[hieuhoang]

ah right. Where's your code? We'll take a look. May give you a hand with it

[XapaJIaMnu]

https://github.com/browsermt/marian-dev/blob/master/src/tensors/cpu/ruy_interface.h

[XapaJIaMnu]

TODO. This file needs to be removed and detecting ARM needs to be done in a different way. The variables that this file sets break the double M1/x86_64 compilation that is supposed to be done on modern OSX.

[vrnmthr]

what are the offending variables in this file?