Commit Graph

2583 Commits

Author SHA1 Message Date
Chip Kerchner
aecb7f9537 Change signature of SBGEMV. 2025-10-07 13:14:20 +00:00
Chip Kerchner
809e1cba8f Better FP16 vectorized GEMV - 20% faster. 2025-10-06 13:19:03 +00:00
Chip Kerchner
e07a9ae418 Merge branch 'develop' into vectorSBGEMV 2025-10-03 17:13:29 +00:00
Chip Kerchner
588f0e87cc Add SBGEMV and SHGEMV routines to RISC-V. 2025-10-03 17:09:16 +00:00
Martin Kroeker
e939c6c315 Merge pull request #5471 from quic/topic/ssymm_direct_sme1
Support for SME1 based ssymm_direct kernel for cblas_ssymm level 3 API
2025-10-01 06:22:36 -07:00
Chip Kerchner
36f9cb85b1 Fix pre-RVV 1.0. 2025-09-30 22:41:31 +00:00
Chip Kerchner
2d82d144e2 Tranverse matrix data in a cache friendly manner for GEMV_N (RISCV). 2025-09-30 21:22:10 +00:00
Martin Kroeker
aaa5c377bc Merge pull request #5465 from ChipKerchner/addRVVVectorizedFP16Packing
Add vectorized packing for FP16 and BF16 for RISC-V.  Reactivate vector packing for FP64 transposed
2025-09-30 09:21:15 -07:00
Rajendra Prasad Matcha
19268471cc Support for SME1 based ssymm_direct kernel for cblas_ssymm level 3 API 2025-09-30 15:05:33 +05:30
Chip Kerchner
67ddda394e Merge branch 'develop' into addRVVVectorizedFP16Packing 2025-09-29 13:49:57 +00:00
pratiklp00
d7b11605d1 fix build issue 2025-09-29 02:02:13 -05:00
Dan Horák
f5ec1c4e53 fix typos in Power8 routines
Fixes: https://github.com/OpenMathLib/OpenBLAS/pull/5448
2025-09-26 16:54:03 +02:00
Dan Horák
681af71d95 drop gcc 15 workaround
As the assembler routines has correctly specified parameter we can drop
the previously applied workaround in https://github.com/OpenMathLib/OpenBLAS/pull/5409.
2025-09-26 16:52:17 +02:00
Chip Kerchner
07d0e742c2 Add vectorized packing for FP16 and BF16. Reactivate vector packing for FP64 transposed. 2025-09-26 14:50:38 +00:00
Martin Kroeker
c92f7f6bb2 Merge pull request #5448 from martin-frbg/issue5372-2
Fix clobber list entries for arrays in POWER kernels that use inline asm
2025-09-26 02:24:50 -07:00
Martin Kroeker
14c9dcaac7 Use generic kernels for SCAL to fix corner cases of Inf/NAN 2025-09-25 20:31:12 +02:00
Chip Kerchner
92f09a6a98 Add BF16 sbgemm on RISCV. 2025-09-22 14:32:43 +00:00
Chip Kerchner
a4abf7828e Fix _Float16 casting issue and reduce LMUL for certain vector instruction from m2 to m1. 2025-09-18 21:30:22 +00:00
Martin Kroeker
1d5279fd29 Fix clobber list entries for arrays in inline asm 2025-09-17 07:02:18 -07:00
学习中的牛马
8b7e4c2b5c Merge branch 'OpenMathLib:develop' into develop 2025-09-15 12:08:17 +08:00
Dayuxiaoshui
2265318d3e Optimize RISC-V RVV omatcopy implementation with latest RVV API\n\nCo-authored-by: gong-flying <gongxiaofei24@iscas.ac.cn> 2025-09-15 11:46:50 +08:00
yuanjia
826cb4588f remove unused variable 2025-09-13 11:35:49 +08:00
yuanjia
53d7452cdf riscv: gemv_t_vector.c optimize 2025-09-13 11:24:49 +08:00
Dayuxiaoshui
bd45b82ed0 Optimize RISC-V RVV omatcopy_ct implementation with advanced vectorization
- Implement block-based memory access optimization (64x64 blocks)
- Add 4-way loop unrolling to reduce loop overhead
- Optimize VSETVL calls to improve vectorization efficiency
- Add software prefetching for better memory access patterns
- Implement fast path for small matrices (<64x64)
- Add cross-compilation script for RISC-V testing
- Improve boundary handling with separate main/tail loops

Co-authored-by: gong-flying <gongxiaofei24@iscas.ac.cn>
2025-09-11 20:01:39 +08:00
Dayuxiaoshui
708d586599 Add OMATCOPY_CT performance test with RVV optimization
Co-authored-by: gong-flying <gongxiaofei24@iscas.ac.cn>
2025-09-11 19:20:26 +08:00
Martin Kroeker
06c09deee9 Merge pull request #5426 from hideaki-motoki/issue5417_axpy_sve
Implementing SVE in `[SD]AXPY` Kernels for `A64FX` and `Graviton3E`
2025-08-26 01:10:14 -07:00
yuanjia
c2cc7a3602 riscv64: optimize gemv_t_vector.c 2025-08-22 16:14:14 +08:00
h-motoki
855945befb Implementing SVE in [SD]AXPY Kernels for A64FX and Graviton3E 2025-08-21 20:56:58 +09:00
Martin Kroeker
9d6df1dd3e Merge pull request #5422 from ChipKerchner/addRVVVectorizedPacking
Add and use vectorized packing in ZVL128B and ZVL256B for RISCV
2025-08-16 13:45:35 -07:00
Martin Kroeker
f3b2a15fad Merge pull request #5420 from yuanjia111/develop
Move the value assignment of vector x in gemv_n_sve.c to the outermos…
2025-08-16 12:06:53 -07:00
Chip Kerchner
64401b4417 Disable vectorized packing for DGEMM - since it is slower than scalar. 2025-08-13 13:41:12 +00:00
Chip Kerchner
c00afc86a6 Add and use vectorized packing to ZVL128B and ZVL256B. Up to 3x+ faster than generic scalar functions. 2025-08-12 17:18:56 +00:00
yuanjia
803e8d4838 Move the value assignment of vector x in gemv_n_sve.c to the outermost loop to reduce the repeated data retrieval.
1.Verify correctness using BLAS-Tester
    2.Using the built-in benchmark to verify performance, the performance of float and doule type improved by about 60% and about 40% respectively.The test command is:
     export OMP_NUM_THREADS=1;numactl -C 10 -l ./sgemv.goto 3000 4000 100
     export OMP_NUM_THREADS=1;numactl -C 10 -l ./dgemv.goto 3000 4000 100
2025-08-12 18:03:16 +08:00
Chris Sidebottom
5f47b872f1 Remove older kernels for BGEMM on NEOVERSEV1 2025-08-11 09:25:19 +00:00
Chris Sidebottom
114316f361 Optimize SBGEMM / BGEMM for NEOVERSEV1 further
This changes the kernels to pack full SVE vectors and reduces the
overall complexity of the inner GEMM loop.
2025-08-11 09:25:13 +00:00
Martin Kroeker
f1ee61ea30 Include NEON header for the bfloat conversion functions 2025-08-04 00:21:39 -07:00
Martin Kroeker
b3ffd5524a Include NEON header for the bfloat conversion functions 2025-08-04 00:20:28 -07:00
Martin Kroeker
0968dddf1a Merge pull request #5409 from martin-frbg/issue5372
Work around gcc15.1 on POWER misoptimizing DGEMV at -O3
2025-07-30 10:36:39 -07:00
Martin Kroeker
a3b9c933c5 mark xbuffer as volatile to work around gcc15.1 optimizer bug 2025-07-30 17:05:36 +02:00
Chip Kerchner
72f082f31d Fix bad vector zero initializer and other compiler warnings for RISC-V. 2025-07-30 14:04:43 +00:00
Martin Kroeker
a5e7c0e3e0 Merge pull request #5396 from abhishek-iitmadras/abhishekk_bfloat16
ARM64: Enable bfloat16 kernels by default
2025-07-28 13:39:08 -07:00
abhishek-fujitsu
0bc79da587 add neon header 2025-07-25 11:10:20 +05:30
Chris Sidebottom
ea2faf0c9a Add optimized BGEMM for NEOVERSEN2 target
This re-uses the existing NEOVERSEN2 8x4 `sbgemm` kernel to implement `bgemm`.
2025-07-24 10:59:28 +00:00
Chris Sidebottom
2c3cdaf74e Optimized BGEMV for NEOVERSEV1 target
- Adds bgemv T based off of sbgemv T kernel
- Adds bgemv N which is slightly alterated to not use Y as an
accumulator due to the output being bf16 which results in loss of
precision
- Enables BGEMM_GEMV_FORWARD to proxy BGEMM to BGEMV with new kernels
2025-07-23 10:51:41 +01:00
Martin Kroeker
e2d941e9af Declare the "small" kernel static in addition to inline 2025-07-22 11:02:32 +02:00
Martin Kroeker
8214700930 Declare the "small" kernel static in addition to inline 2025-07-22 11:01:37 +02:00
Martin Kroeker
39c90f9859 Merge pull request #5380 from quic/topic/sgemm_direct_sme1_alpha_beta
SME1 based direct kernel (with alpha and beta) for cblas_sgemm level 3
2025-07-18 23:23:39 +02:00
Rajendra Prasad Matcha
eae0abfdb6 SME1 based direct kernel with alpha and beta for cblas_sgemm level 3 API. 2025-07-17 16:14:31 +05:30
Chris Sidebottom
947d7af4c9 Fix CMake references to bscal and bgemv 2025-07-15 15:41:53 +01:00
Chris Sidebottom
e105411460 Add infrastructure for bgemv/bscal
- Sets up all the various entrypoints for `bgemv`
- Adds `bscal` for use in the `bgemv` interface
- Adds test cases for comparing `sgemv` and `bgemv`
- Adds generic kernels for `bgemv_n` and `bgemv_t` which are accurate
enough to pass above tests
2025-07-15 14:48:57 +01:00