OpenMathLib/OpenBLAS
1
0
Fork 0
mirror of https://github.com/OpenMathLib/OpenBLAS synced 2026-06-12 00:41:42 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
Files
114316f36102e3500ec84c055acb37802afdb313
OpenBLAS/kernel/arm64
History
Chris Sidebottom 114316f361 Optimize SBGEMM / BGEMM for NEOVERSEV1 further 
This changes the kernels to pack full SVE vectors and reduces the
overall complexity of the inner GEMM loop.
2025-08-11 09:25:13 +00:00
..
amax.S
…
asum.S
…
axpy.S
…
bgemm_beta_neon.c
Add optimized BGEMM kernel for NEOVERSEV1 target
2025-07-10 23:23:27 +00:00
bgemm_kernel_2vlx4_neoversev1_impl.c
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
bgemm_kernel_2vlx4_neoversev1.c
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
bgemm_kernel_4x4_neoversev1_impl.c
add neon header
2025-07-25 11:10:20 +05:30
bgemm_kernel_4x4_neoversev1.c
add neon header
2025-07-25 11:10:20 +05:30
bgemm_ncopy_2vl_neoversev1.c
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
bgemm_ncopy_4_neoversev1.c
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
bgemm_tcopy_2vl_neoversev1.c
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
bgemm_tcopy_4_neoversev1.c
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
bgemv_n_sve_v3x4.c
add neon header
2025-07-25 11:10:20 +05:30
casum_thunderx2t99.c
…
casum.S
…
cgemm_kernel_4x4.S
…
cgemm_kernel_8x4_cortexa53.c
optimize cgemm on ARM cortex A53 & cortex A55
2021-12-12 17:22:52 +08:00
cgemm_kernel_8x4_thunderx2t99.S
Move ALPHA_I out of register 18 (reserved on OSX)
2023-04-13 18:00:47 +02:00
cgemm_kernel_8x4.S
move ALPHA_I out of register 18 (reserved on OSX)
2023-04-13 17:59:48 +02:00
cgemm_kernel_sve_v1x4.S
Use SVE zgemm/cgemm on Arm(R) Neoverse(TM) V1 core
2023-07-27 14:12:20 +01:00
cgemm_ncopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
cgemm_tcopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
copy_thunderx2t99.c
[WIP] Work around assembler limitations in current LLVM for Windows on Arm (#5076)
2025-01-18 16:45:56 +01:00
copy.S
…
csum_thunderx2t99.c
add csum/zsum kernels (trivially derived from the asum ones)s)
2024-02-25 17:55:36 +01:00
csum.S
…
ctrmm_kernel_4x4.S
…
ctrmm_kernel_8x4.S
Move ALPHA_I out of register 18 (reserved on OSX)
2023-04-13 18:03:35 +02:00
ctrmm_kernel_sve_v1x4.S
add cgemm ctrmm sve kernels
2022-01-05 09:09:18 +01:00
dasum_thunderx2t99.c
[WIP] Work around assembler limitations in current LLVM for Windows on Arm (#5076)
2025-01-18 16:45:56 +01:00
daxpy_thunderx2t99.S
…
daxpy_thunderx.c
…
ddot_thunderx.c
…
dgemm_beta.S
…
dgemm_kernel_4x4_cortexa53.c
MOD: optimize normal DGEMM on ARMV8 cortex-A53 & cortex-A55
2021-11-18 21:14:43 +08:00
dgemm_kernel_4x4.S
…
dgemm_kernel_4x8.S
…
dgemm_kernel_8x4_thunderx2t99.S
…
dgemm_kernel_8x4.S
…
dgemm_kernel_sve_v1x8.S
some clean-up & commentary
2021-11-21 14:56:27 +01:00
dgemm_kernel_sve_v2x8.S
Remove prefetches from SVE kernels
2022-12-16 14:43:09 +00:00
dgemm_ncopy_4.S
…
dgemm_ncopy_8.S
…
dgemm_small_kernel_nn_sve.c
Better header guard around bridge
2024-07-20 14:39:57 +01:00
dgemm_small_kernel_nt_sve.c
Better header guard around bridge
2024-07-20 14:39:57 +01:00
dgemm_small_kernel_tn_sve.c
small gemm kernel packing modifications
2025-01-23 09:41:45 +05:30
dgemm_small_kernel_tt_sve.c
small gemm kernel packing modifications
2025-01-23 09:41:45 +05:30
dgemm_tcopy_4.S
…
dgemm_tcopy_8.S
Remove unused TEMP2 and reshuffle to leave x18 unused (reserved on OSX)
2021-09-17 09:18:25 +02:00
dot_kernel_asimd.c
Accumulate results in output register explicitly
2025-06-09 19:03:22 +00:00
dot_kernel_sve_v8.c
Performance improvements of [SD]DOT with loop-unrolling on A64FX
2025-07-04 22:57:44 +09:00
dot_kernel_sve.c
add clobber list
2024-06-14 22:07:44 +02:00
dot_sve_v8.c
Performance improvements of [SD]DOT with loop-unrolling on A64FX
2025-07-04 22:57:44 +09:00
dot_thunderx.c
…
dot.c
Performance improvements of [SD]DOT with loop-unrolling on A64FX
2025-07-04 22:57:44 +09:00
dot.S
…
dtrmm_kernel_4x4.S
…
dtrmm_kernel_4x8.S
…
dtrmm_kernel_8x4.S
Move temp to x21 to leave x18 unused (reserved on OSX)
2021-09-17 09:24:11 +02:00
dtrmm_kernel_sve_v1x8.S
some clean-up & commentary
2021-11-21 14:56:27 +01:00
dznrm2_thunderx2t99_fast.c
…
dznrm2_thunderx2t99.c
remove another early exit for incx < 0
2024-03-12 18:47:00 +01:00
gemm_ncopy_complex_sve_v1x4.c
Use SVE zgemm/cgemm on Arm(R) Neoverse(TM) V1 core
2023-07-27 14:12:20 +01:00
gemm_ncopy_sve_v1x8.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
gemm_small_kernel_permit_sve.c
Remove k2 loop from DGEMM TN and use a more conservative heuristic for SGEMM
2024-07-18 17:37:18 +01:00
gemm_tcopy_complex_sve_v1x4.c
Use SVE zgemm/cgemm on Arm(R) Neoverse(TM) V1 core
2023-07-27 14:12:20 +01:00
gemm_tcopy_sve_v1x8.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
gemv_n_sve_v1x3.c
Optimize gemv_n_sve_v1x3 kernel
2025-06-11 10:16:56 +00:00
gemv_n_sve_v4x3.c
fixed a potential out-of-bounds on gemv.
2025-04-22 19:56:44 +09:00
gemv_n_sve.c
Optimize gemv_n_sve kernel
2025-03-10 16:39:20 +00:00
gemv_n.S
…
gemv_t_sve_v1x3.c
Simplify gemv_t_sve_v1x3 kernel
2025-01-21 13:40:17 +00:00
gemv_t_sve_v4x3.c
Loop-unrolled transposed [SD]GEMV kernels for A64FX and Neoverse V1
2024-12-02 18:46:00 +09:00
gemv_t_sve.c
Add accumulators to AArch64 GEMV Kernels
2024-07-31 13:09:14 +01:00
gemv_t.S
Add accumulators to AArch64 GEMV Kernels
2024-07-31 13:09:14 +01:00
iamax_thunderx2t99.c
…
iamax.S
…
izamax_thunderx2t99.c
…
izamax.S
…
KERNEL
Further rearranged the rotm kernel for the different architectures.
2025-01-22 11:41:12 +08:00
KERNEL.A64FX
Performance improvements of [SD]DOT with loop-unrolling on A64FX
2025-07-04 22:57:44 +09:00
KERNEL.AMPERE1
reduce duplicate kernel code
2025-06-17 03:05:34 -04:00
KERNEL.ARMV8
…
KERNEL.ARMV8SVE
Use SVE kernel for S/DGEMVN for SVE machines
2025-04-22 09:40:13 +00:00
KERNEL.ARMV9SME
Support for SME1 based sgemm_direct kernel for cblas_sgemm level 3 API
2025-02-13 14:51:21 +05:30
KERNEL.CORTEXA53
optimize cgemm on ARM cortex A53 & cortex A55
2021-12-12 17:22:52 +08:00
KERNEL.CORTEXA55
Reduce duplication in kernel definitions
2023-12-23 12:39:53 +00:00
KERNEL.CORTEXA57
…
KERNEL.CORTEXA72
…
KERNEL.CORTEXA73
…
KERNEL.CORTEXA76
Add support for Cortex-A76
2024-04-02 19:41:44 +02:00
KERNEL.CORTEXA510
Fix outdated SVE kernel definitions for Cortex cpus by aliasing to ARMV8SVE
2023-11-03 14:55:31 +01:00
KERNEL.CORTEXA710
Fix outdated SVE kernel definitions for Cortex cpus by aliasing to ARMV8SVE
2023-11-03 14:55:31 +01:00
KERNEL.CORTEXX1
CortexX1 is ARMV8 like A7x
2022-03-28 17:28:29 +02:00
KERNEL.CORTEXX2
Fix outdated SVE kernel definitions for Cortex cpus by aliasing to ARMV8SVE
2023-11-03 14:55:31 +01:00
KERNEL.EMAG8180
…
KERNEL.FALKOR
…
KERNEL.FT2000
Add initial support for Phytium FT2000 series and ARMV9 Cortex 510/710/X1/X2
2022-03-27 15:29:20 +02:00
KERNEL.generic
Further rearranged the rotm kernel for the different architectures.
2025-01-22 11:41:12 +08:00
KERNEL.NEOVERSEN1
Merge pull request #5225 from annop-w/gemv_n
2025-04-17 01:54:10 -07:00
KERNEL.NEOVERSEN2
Add optimized BGEMM for NEOVERSEN2 target
2025-07-24 10:59:28 +00:00
KERNEL.NEOVERSEV1
Optimize SBGEMM / BGEMM for NEOVERSEV1 further
2025-08-11 09:25:13 +00:00
KERNEL.NEOVERSEV2
Add optimized BGEMM for NEOVERSEN2 target
2025-07-24 10:59:28 +00:00
KERNEL.THUNDERX
…
KERNEL.THUNDERX2T99
Add SVE implementation for sdot/ddot
2022-12-01 12:07:50 +00:00
KERNEL.THUNDERX3T110
Reduce duplication in kernel definitions
2023-12-23 12:39:53 +00:00
KERNEL.TSV110
…
KERNEL.VORTEX
Use Neoverse's current mix of ThunderX2 kernels for Vortex as well
2021-10-06 11:06:43 +02:00
Makefile
…
nrm2.S
…
rot_kernel_c.c
Added Updated swap and rot sve kernels.
2024-12-13 11:15:29 +05:30
rot_kernel_sve.c
Added Updated swap and rot sve kernels.
2024-12-13 11:15:29 +05:30
rot.c
Added Updated swap and rot sve kernels.
2024-12-13 11:15:29 +05:30
rot.S
…
sasum_thunderx2t99.c
[WIP] Work around assembler limitations in current LLVM for Windows on Arm (#5076)
2025-01-18 16:45:56 +01:00
sbgemm_beta_neoversen2.c
neoverse n2 sbgemm: init file
2022-06-29 10:14:21 +08:00
sbgemm_beta_neoversev1.c
* checkpoint sbgemm for SVE-256
2025-02-03 12:49:49 +00:00
sbgemm_kernel_4x4_neoversev1_impl.c
optimized sbgemm kernel for neoverse-v1 (sve-256)
2025-02-05 10:06:37 +00:00
sbgemm_kernel_4x4_neoversev1.c
optimized sbgemm kernel for neoverse-v1 (sve-256)
2025-02-05 10:06:37 +00:00
sbgemm_kernel_8x4_neoversen2_impl.c
Include NEON header for the bfloat conversion functions
2025-08-04 00:21:39 -07:00
sbgemm_kernel_8x4_neoversen2.c
Include NEON header for the bfloat conversion functions
2025-08-04 00:20:28 -07:00
sbgemm_ncopy_4_neoversen2.c
Change file name to match the norm and delete useless code.
2022-10-28 17:09:39 +08:00
sbgemm_ncopy_4_neoversev1.c
optimized sbgemm kernel for neoverse-v1 (sve-256)
2025-02-05 10:06:37 +00:00
sbgemm_ncopy_8_neoversen2.c
bugfix for sbgemm_ncopy_8_neoversen2
2022-12-05 05:10:18 -05:00
sbgemm_tcopy_4_neoversen2.c
Add sbgemm_ncopy_8 and sbgemm_tcopy_4
2022-11-29 04:46:14 -05:00
sbgemm_tcopy_4_neoversev1.c
Update sbgemm_tcopy_4_neoversev1 kernel to use standard C types
2025-06-19 14:26:16 +01:00
sbgemm_tcopy_8_neoversen2.c
Improve the performance of sbgemm_tcopy on neoversen2
2022-11-28 04:17:54 -05:00
sbgemv_n_neon.c
fix bugs in aarch64 sbgemv_n kernel
2025-03-14 17:55:40 +00:00
sbgemv_t_bfdot.c
Optimized BGEMV for NEOVERSEV1 target
2025-07-23 10:51:41 +01:00
scal.S
make NAN handling depend on the dummy2 parameter
2024-07-17 23:24:19 +02:00
scnrm2_thunderx2t99.c
remove another early exit for incx < 0
2024-03-12 18:49:27 +01:00
sgemm_beta.S
Fix file permissions (issue 4095)
2023-07-23 20:37:07 +02:00
sgemm_direct_alpha_beta_arm64_sme1.c
SME1 based direct kernel with alpha and beta for cblas_sgemm level 3 API.
2025-07-17 16:14:31 +05:30
sgemm_direct_arm64_sme1.c
Fix ARMV9SME target in DYNAMIC_ARCH and add SME query code for MacOS (#5222)
2025-05-10 22:39:32 +02:00
sgemm_direct_sme1_preprocess.S
Support for SME1 based sgemm_direct kernel for cblas_sgemm level 3 API
2025-02-13 14:51:21 +05:30
sgemm_direct_sme1.S
Support for SME1 based sgemm_direct kernel for cblas_sgemm level 3 API
2025-02-13 14:51:21 +05:30
sgemm_kernel_4x4.S
…
sgemm_kernel_8x8_cortexa53.S
…
sgemm_kernel_8x8.S
…
sgemm_kernel_16x4_thunderx2t99.S
…
sgemm_kernel_16x4.S
…
sgemm_kernel_sve_v1x8.S
add sgemm kernel and copy functions for sgemm and ssymm
2021-11-28 18:12:47 +01:00
sgemm_kernel_sve_v2x8.S
revert "move alpha out of register 18" (out of PR scope, no SVE on Apple hw)
2023-04-17 14:23:13 +02:00
sgemm_ncopy_4.S
Optimize aarch64 sgemm_ncopy
2025-03-13 10:17:43 +00:00
sgemm_ncopy_8.S
Optimize aarch64 sgemm_ncopy
2025-03-13 10:17:43 +00:00
sgemm_small_kernel_nn_sve.c
Better header guard around bridge
2024-07-20 14:39:57 +01:00
sgemm_small_kernel_nt_sve.c
Better header guard around bridge
2024-07-20 14:39:57 +01:00
sgemm_small_kernel_tn_sve.c
small gemm kernel packing modifications
2025-01-23 09:41:45 +05:30
sgemm_small_kernel_tt_sve.c
small gemm kernel packing modifications
2025-01-23 09:41:45 +05:30
sgemm_tcopy_8.S
…
sgemm_tcopy_16.S
change line endings from CRLF to LF
2022-11-16 22:24:01 +01:00
sgemv_n_neon.c
Improve performance for SGEMVN on NEONVERSEN1
2025-04-16 09:54:30 +00:00
sme_abi.h
SME1 based direct kernel with alpha and beta for cblas_sgemm level 3 API.
2025-07-17 16:14:31 +05:30
strmm_kernel_4x4.S
…
strmm_kernel_8x8_cortexa53.S
…
strmm_kernel_8x8.S
…
strmm_kernel_16x4.S
Move temp to x21 to leave x18 unused (reserved on OSX)
2021-09-17 09:28:19 +02:00
strmm_kernel_sve_v1x8.S
strmm sve v1x8 kernel
2021-12-05 14:03:08 +01:00
sum.S
…
swap_kernel_c.c
Added Updated swap and rot sve kernels.
2024-12-13 11:15:29 +05:30
swap_kernel_sve.c
Update swap_kernel_sve.c
2024-12-13 16:47:58 +05:30
swap_thunderx2t99.S
…
swap.c
Added Updated swap and rot sve kernels.
2024-12-13 11:15:29 +05:30
swap.S
…
symm_lcopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
symm_ucopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
symv_L_asimd_4x4.c
Add symv kernels for arm64
2025-04-11 20:39:52 +09:00
symv_L_sve_v1x4.c
Add symv kernels for arm64
2025-04-11 20:39:52 +09:00
symv_microk_asimd_4x4.c
Add symv kernels for arm64
2025-04-11 20:39:52 +09:00
symv_microk_sve_v1x4.c
Add symv kernels for arm64
2025-04-11 20:39:52 +09:00
symv_U_asimd_4x4.c
Add symv kernels for arm64
2025-04-11 20:39:52 +09:00
symv_U_sve_v1x4.c
Add symv kernels for arm64
2025-04-11 20:39:52 +09:00
trmm_lncopy_sve_v1.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
trmm_ltcopy_sve_v1.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
trmm_uncopy_sve_v1.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
trmm_utcopy_sve_v1.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
trsm_kernel_LN_sve.c
add sve ztrsm
2022-01-15 22:27:25 +01:00
trsm_kernel_LT_sve.c
add sve ztrsm
2022-01-15 22:27:25 +01:00
trsm_kernel_RN_sve.c
add sve ztrsm
2022-01-15 22:27:25 +01:00
trsm_kernel_RT_sve.c
add sve ztrsm
2022-01-15 22:27:25 +01:00
trsm_lncopy_sve.c
Work around Clang failing to disambiguate SVE intrinsics and add AppleClang crossbuild to MacOS/arm64 DYNAMIC_ARCH in AzureCI (#4140)
2023-07-14 11:06:48 +02:00
trsm_ltcopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
trsm_uncopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
trsm_utcopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
zamax.S
…
zasum_thunderx2t99.c
[WIP] Work around assembler limitations in current LLVM for Windows on Arm (#5076)
2025-01-18 16:45:56 +01:00
zasum.S
…
zaxpy.S
…
zdot_thunderx2t99.c
Add a clobber list to fix utest errors seen with gcc13 on Apple M
2024-06-20 16:19:32 +02:00
zdot.S
…
zgemm_kernel_4x4_cortexa53.c
MOD: add comments to a53 zgemm kernel
2021-11-25 22:48:48 +08:00
zgemm_kernel_4x4_thunderx2t99.S
…
zgemm_kernel_4x4.S
move alpha to x19/x20 to leave x18 unused for OSX
2021-09-17 09:42:17 +02:00
zgemm_kernel_sve_v1x4.S
Use SVE zgemm/cgemm on Arm(R) Neoverse(TM) V1 core
2023-07-27 14:12:20 +01:00
zgemm_ncopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
zgemm_tcopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
zgemv_n.S
…
zgemv_t.S
…
zhemm_ltcopy_sve.c
Fix ZHEMM copy for SVE
2023-07-27 13:27:28 +01:00
zhemm_utcopy_sve.c
Fix ZHEMM copy for SVE
2023-07-27 13:27:28 +01:00
znrm2.S
…
zrot.S
…
zscal.S
fix handling of dummy2
2025-06-12 03:03:01 -07:00
zsum_thunderx2t99.c
add csum/zsum kernels (trivially derived from the asum ones)s)
2024-02-25 17:55:36 +01:00
zsum.S
…
zsymm_lcopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
zsymm_ucopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrmm_kernel_4x4.S
Move alphaI to x22 to leave x18 unused (reserved on OSX)
2021-09-17 09:53:18 +02:00
ztrmm_kernel_sve_v1x4.S
fix sve ztrmm kernel
2022-01-04 14:42:07 +01:00
ztrmm_lncopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrmm_ltcopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrmm_uncopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrmm_utcopy_sve_v1.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrsm_lncopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrsm_ltcopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrsm_uncopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00
ztrsm_utcopy_sve.c
Disambiguate whilelt
2023-07-25 20:15:44 +01:00