6,822 questions
Tooling
0
votes
6
replies
52
views
Vectorization PNG image with precise color number
I have a task to convert user PNG image to SVG with maximum possible color number coming from user setting (e.g. 8). It means that i should make initially image created from 8 colors and then convert ...
- 235
3
votes
1
answer
201
views
How to figure out why Rust failed to vectorize and fix it?
How do I know if rustc or LLVM is doing the vectorizing?
How can I fix this specific example?
Here's a simplified piece of extremely performance-sensitive decompression code I would like to use:
pub ...
- 1,096
3
votes
2
answers
190
views
Why do GCC and Clang fail to auto-vectorize simple loop?
I have two functions counting the occurrences of a target char in the given input buffer. The functions vary only in how they communicate the result back to the caller; one returns the result and the ...
- 193
2
votes
2
answers
135
views
Applying a function to two matrices column-by-column
Short version: How do I vectorize two matrices A and B by column so that I can pass the 1st, 2nd, ..., ith column of A and the 1st, 2nd, ..., ith column of B to a function without a loop?
I have two ...
- 358
5
votes
1
answer
289
views
Vectorized `grep` for multiple patterns?
(Edited to incorporate fixed=TRUE as suggested by @r2evans.)
For two character vectors, str and pattern, I want to return the indices where pattern is found in str, as in this Q&A:
pattern <- c(...
- 17.5k
3
votes
1
answer
88
views
How to vectorize R plotmath constructions
In this example, I want to construct expressions to vectorize the labelling of three ellipses in the following fig with expressions for the size of the ellipse, but each expression involves data ...
- 4,083
1
vote
2
answers
164
views
Are JAX operations already vectorized?
In the documentation, JAX provides vectorization. However, aren't JAX operations already vectorized? For example, to add two vectors, I thought that the element-wise additions were vectorized ...
- 161
5
votes
2
answers
270
views
Why is Numba more efficient with 2D vs 1D version of this loop?
I'm using Python 3.12.11 with Numba 0.61.2 on Ubuntu 22.04.5 and AMD Ryzen 7 3800X CPU. This benchmark:
import numpy as np, timeit as ti, numba as nb
@nb.njit(fastmath=True)
def f1d(img, w):
fl = ...
- 8,628
0
votes
0
answers
154
views
Is there a way to monitor progress of vmapped function in JAX (like with tqdm)?
Let's say I have an expensive jitted function f(x) and I've vmapped it over a large array of inputs -- is there any way to monitor the progress of how many of the vmapped inputs are done being ...
- 173
0
votes
0
answers
105
views
RISC-V vs C Code Comparison for Simple Multiply and Accumulate (MAC) Operation
We tried profiling a simple MAC operation using both RISC-V Vector (RVV) intrinsics and plain C code. Surprisingly, the C version performs better, even though the intrinsics code processes 16 ...
- 1
0
votes
1
answer
79
views
MUVERA compression not working with text2colbert-jinaai in Weaviate
I'm trying to use MUVERA compression with Jina ColBERT v2 embeddings in Weaviate, following the official documentation. However, MUVERA compression is not being applied: I'm still getting raw multi-...
- 351
3
votes
2
answers
110
views
How to vectorize computation of spearman correlation between a 2D array and a reference 1D array with scipy
I have a matrix M with size (37, N) and an additionnal 1D reference vector of size (37,1)
I am looking for a way to compute the spearman correlation between each sample of M and my reference to obtain ...
4
votes
1
answer
75
views
How can I vectorize a function that returns eigenvalues and eigenvectors of a matrix in python?
I'm working with a function in Python that constructs a 4×4 matrix based on inputs (x1, y1, x2, y2), and computes its eigenvalues and eigenvectors using np.linalg.eigh.
Here is a simplified version of ...
1
vote
1
answer
90
views
Speeding up 3D interpolation
I am using python to perform the homotopy formula to obtain a vector potential A of a magnetic field B at points x in 3D space:
A(x) = integral_L=0,1 (B(Lx)cross(Lx)dL)
where cross represents the ...
- 29
4
votes
2
answers
184
views
How to remove and speed up a for loop over matrix columns by vectorisation?
Intro
Consider a tournament with four players and the following schedule:
Rd 1: 1-4, 2-3.
Rd 2: 4-3, 1-2.
Rd 3: 2-4, ---.
The usual way to display tournament results is the cross table, for example:
...
- 1,602
1
vote
1
answer
130
views
How to set a boolean flag True after entry and back to False after exit using Pandas (vectorized)?
I'm working with a Pandas DataFrame where I need to track a boolean flag (in_position)
that becomes True after an entry_signal and resets to False only after an exit_signal.
This must be done ...
2
votes
2
answers
88
views
Using vectors to manipulate columns, and add new ones, in multiindex dataframes
I have a dataframe with a column multi-index, df1, with a datetime index and 2 levels: level 0, called Capitals, has columns A, B, C, and level 1, called Smalls, has columns a, b, c, d, e.
Capitals
A
...
- 167
2
votes
1
answer
100
views
Python Vectorized Mask Generation (Numpy) [closed]
I have an arbitrary Matrix M which is (N x A). I have a column vector V (N x 1) which has on each row the amount of entries I would like to keep from the original M <= A (starting from the leftmost)...
1
vote
1
answer
107
views
How to populate a 2-d numpy array with values from a third dimension?
New Post: Processing satellite conjunctions with numpy efficiently
Original Post:
I have a numpy array of shape n x m x r, where the n axis represents an object, the m axis represents a timestep and ...
- 45
1
vote
1
answer
173
views
Using vmap to parallelize multiple models input in JAX / Flax.nnx
I have a function that is roughly as follows
from flax import nnx
from jax import Array
from typing import List
def predict(models: List[nnx.Module], imgs: Array):
for i, agent in enumerate(...
- 21
0
votes
1
answer
72
views
Write Timeout for Batch Vectorization in Docker – TLS & Container Restart Issues
I'm building a chatbot that uses Weaviate and t2v-transformers for vectorizing my knowledge base. My way of sends batches like this:
var batchResponse = await _httpClient.PostAsync($"{...
- 633
0
votes
1
answer
89
views
Vectorizing a Battery SOC Update with Recursive Charge/Discharge Constraints in NumPy
I have a battery simulation script where the battery’s state-of-charge (SOC) is updated iteratively. The charge and discharge values (how much energy is added/taken from the battery) at each timestep ...
3
votes
1
answer
240
views
why is my simd vector plus and set slower than using std::transform and std::plus<T> - am i doing my simd wrong?
New to SIMD please go easy on me if I have made any mistakes.
I am using windows vs studio for dev, msvc ISO C++20. My processor is 11th Gen Intel(R) Core(TM) i7-11370H @ 3.30GHz
Before using AXV, I ...
2
votes
2
answers
281
views
System.Numerics.Vector<T> or System.Runtime.Intrinsics.Vector128<T>
The documentation for both Vector and Vector128 structs are really similar and there is an AsVector/AsVector128 method to switch between them.
However, is there a case where I should use one over the ...
0
votes
1
answer
27
views
reduction/conjuction/disjunction functions for OpenCL vector types?
OpenCL offers built-in/intrinsic "vector types" (see table 3 at the link), such as int4 or float2. It also defines binary and unary elementwise operators which accept these types, e.g. ...
- 137k
4
votes
1
answer
113
views
need to vectorize efficiently calculating only certain values in the matrix multiplication A * B, using a logical array L the size of A * B
I have matrices A (m by v) and B (v by n). I also have a logical matrix L (m by n).
I am interested in calculating only the values in A * B that correspond to logical values in L (values of 1s). ...
- 41
2
votes
4
answers
450
views
Jax numpy extracting non-nan values gives NonConcreteBooleanIndexError
I have a jax 2d array with some nan-values
array_2d = jnp.array([
[jnp.nan, 1, 2, jnp.nan, 3],
[10 ,jnp.nan, jnp.nan, 20,jnp.nan]
])
and want to get an ...
- 1,263
2
votes
1
answer
118
views
Is there a smart way to vectorize a nested for-loop where the inner index is limited by the outer index?
Is there a smart way to vectorize a nested for loop of inner products, where the inner index is lower bound by the outer index?
Here's a simple example. Say that arr1 and arr2 are numpy arrays each ...
- 49
2
votes
0
answers
67
views
Swig attempts to cast numpy.float64 to C++ double for new C++ functions
I am working on a Python project that uses swig to connect the main Python code to a C++ module. I recently added a new function to the C++ library, but in functions with the @numpy.vectorize ...
- 21
2
votes
1
answer
98
views
Is GCC/Clang able to auto-vectorize std::inner_product?
I have the following code:
#include <iostream>
#include <numeric>
int main() {
volatile float
a0[4] = {1, 2, 3, 4},
a1[4] = {4, 5, 6, 7};
std::cout << std::...
- 5,110
-5
votes
1
answer
194
views
Why is GCC not autovectorising this code unless I expicitly specify any of the possible cost model?
I have a little minimal sample algorithm (please ignore if the algorithm itself doesn't make sense and could be changed to be different, its just a contrived sample to demonstrate what I'm seeing).
...
- 684
0
votes
0
answers
29
views
vectorize returns 0s instead of results
The MWE below shows that vectorize returns zeros instead of the actual values. Compare the printout from within the function to the arrays returned in the second test. It is curious that the first ...
- 21
0
votes
1
answer
67
views
How to vectorize my code in pandas? it is long and inefficient
I have this code that I wrote and it's taking too long to run. I was advised to vectorize this operation but so far I have found only multiplication examples. Here is my code:
my_dict = {}
for i in ...
8
votes
4
answers
177
views
How to divide a matrix in MATLAB into N^2 segments each with NxN elements?
Assuming we have a matrix M of size N^2 x N^2 elements (e.g., 9x9), what's the fastest way to split it into say 3x3 segments (each with 3x3 elements).
One way that comes to mind is the following:
M = ...
- 197
0
votes
1
answer
81
views
How to vectorize Pandas DateTimeIndex
I'm trying to avoid a for loop with DateTimeIndex. I have a function get_latest that looks up the most recent wage index value. When I step through the dates of pay days, the lookup works fine. When I ...
0
votes
2
answers
149
views
Vectorizing three nested loops that calculate the daily mean of hourly data
Is there a way to vectorize the following three-nested loop that calculate the daily mean of hourly data? The function below loops first over the year, then months, and finally over days. It also ...
- 725
0
votes
1
answer
65
views
Python: minimize function in respect to i-th variable
I have a function func(x) where the argument is a vector of length n. I would like to minimize it in respect to i-th component of x while keeping the other components fixed. So to express it as a ...
- 803
3
votes
1
answer
144
views
How to extract sub arrays from a larger array with two start and two stop 1-D arrays in Python?
I am looking for a way to vectorize the following code,
# Let cube have shape (N, M, M)
sub_arrays = np.empty(len(cube), 3, 3)
row_start = ... # Shape (N,) and are integers in range [0, M-2]
row_end ...
- 33
2
votes
1
answer
94
views
AVX2 / gcc: Improve CPU-level parallelism by using different registers
I have this code:
__attribute__((target("avx2")))
size_t lower_than_16(const uint64_t values[16], uint64_t x)
{
__m256i vx = _mm256_set1_epi64x(x);
__m256i vvals1 = ...
- 2,644
4
votes
5
answers
271
views
Efficiently draw random samples without replacement from an array in python
I need to draw random samples without replacement from a 1D NumPy array. However, performance is critical since this operation will be repeated many times.
Here’s the code I’m currently using:
import ...
- 87
1
vote
1
answer
108
views
Vectorize objects in python Jax
I am unsure what is the best way to vectorize objects in Python Jax.
In particular, I want to write a code that handles both calling a method from a single instantiation of a class and from multiple (...
- 324
0
votes
0
answers
53
views
putting mapply in loop "causes the condition has length > 1" error
I am trying to be more elegant in my coding and leveraging the apply functions. I wanted to stick one in a loop like so
for (ind in (ind2fix ))
{
ADPPK[,ind] = mapply(categorize_cov,value = ADPPK[,...
- 681
1
vote
1
answer
65
views
How to Vectorize Timezone Offset Calculation in Pandas DatetimeIndex for Improved Performance?
I'm working with a Pandas DataFrame that utilizes a DatetimeIndex with timezone information. My objective is to compute the timezone offset (in hours) for each timestamp and store these offsets in a ...
- 13
0
votes
1
answer
91
views
Manipulation of a Pandas dataframe most time- and memory-efficiently
Please imagine I have a dataframe like this:
df = pd.DataFrame(index=pd.Index(['1', '1', '2', '2'], name='from'), columns=['to'], data= ['2', '2', '4', '5'])
df:
Now, I would like to calculate a ...
- 2,151
1
vote
1
answer
245
views
vectorize.upstream_error - failed to parse upsert vectors request in ndjson format: line Some(0) was not expected format
I am trying to upsert a vector in my cloudflare vectorize index
For now I am testing from my terminal using cURL
Call I am making is this:
curl -v --request POST --url https://api.cloudflare.com/...
- 11
0
votes
1
answer
45
views
Summarize higher dimensions in numpy
I have a numpy array that holds board game states for all possible moves, and I want to summarize some of those moves. I'm struggling to vectorize that code and avoid a for loop when I choose which ...
- 56.7k
0
votes
0
answers
97
views
Vectorizing irregular and non SIMD register width multiple loop
Assume the function below. There are two extents that have been picked to be 3 and 17. We wish to vectorize SomeWork (it is in the translation unit and simple).
The naive approach I take is to flatten ...
- 715
0
votes
0
answers
11
views
Using which() with apply [duplicate]
I have two vectors, say
x <- c("a","b","c","d","e")
y <- c("b","e")
I want to derive a vector containing the indices of ...
- 1
1
vote
1
answer
89
views
How to efficiently compute and process 3x3x3 voxel neighborhoods in a 3D NumPy array?
I am working on a function to process 3D images voxel-by-voxel. For each voxel, I compute the difference between the voxel value and its 3x3x3 neighborhood, apply distance-based scaling, and determine ...
- 31
3
votes
2
answers
339
views
Speeding Up Monte Carlo Simulations for Weather-Dependent Claims
I’m working on a Monte Carlo simulation project, and I need help optimizing some aspects of the problem. Here’s the scenario :
We model claims R (linked to weather-dependent events) as independent ...
- 143