You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm researching on different libraries for big data, and I came across this one.
One of the benchmarks I need to do is creating pytorch DataLoader class with the standard library dataset (in this case, it would be vaex DataFrame) and measuring time it takes to get several random samples from the DataLoader.
Another library I came across is Uber Petastorm. The interesting thing they did is implementing API compatible with TensorFlow and Torch. (I still can't get it to work due to dependency hell tho :'((( )
So, how about implementing something like lazy random sampler and other API methods for those two libraries in vaex as well?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I'm researching on different libraries for big data, and I came across this one.
One of the benchmarks I need to do is creating pytorch DataLoader class with the standard library dataset (in this case, it would be vaex DataFrame) and measuring time it takes to get several random samples from the DataLoader.
Another library I came across is Uber Petastorm. The interesting thing they did is implementing API compatible with TensorFlow and Torch. (I still can't get it to work due to dependency hell tho :'((( )
So, how about implementing something like lazy random sampler and other API methods for those two libraries in vaex as well?
Beta Was this translation helpful? Give feedback.
All reactions