Is there any way to prevent the c3-remote connection from timing out in Jupyter for a long-time execution, like overnight or over a weekend?
I don’t believe that would be easy:
- timeout could be due to network connection/load balancer (whose maximum either we don’t have control over or can’t be infinite)
- timeout could be due to auth token expiring, which is desirable
I think alternatively, another way we could think about the problem is how to execute whatever action you’re trying to execute asynchronously (i.e. like a Job). That way, there isn’t a long running connection to the c3server. For this, you can look at using the
InvalidationQueue and corresponding jobs like
BatchJob, or the light-weight
In distributed systems, you have to assume that any resource could die at any time. For this reason, it is very bad practice to write a method that takes several days to run.
Dennis is giving you the correct way to think about this. If you can provide your use case we can try to help you achieve a better architecture