Posts

Showing posts with the label Big data

Big data for training models in the cloud

Image
Big data for training models in the cloud When you have a lot of data like you can't run training on your local machine, or the size of that data is bigger than your hard drive, it's time to look at other options. To the cloud One concrete option is to transfer machine learning training to another computer to access additional storage, thus freeing up your hard drive space and allowing you to work on other things while the pieces of training are taking place. Let's break down some of the parts that need to be moved to the cloud. It is useful to think of our training as a need for two primary resources: calculation and storage. The interesting thing here is that we don't have to tie them so tightly as you expected before. We can decode them, which means we can take advantage of special systems for both of us. This can affect data efficiency when dealing with big data. Compute loads are easily moved together, but moving large datasets can be a bit more engaging. However,