A DeliveryStore is a place to put the files from deliveries. In more technical terms, it is a place where each file related to a devilry.apps.core.models.FileMeta is stored.
Devilry on comes with one DeliveryStore ready for production use, FsDeliveryStore. To enable a DeliveryStore, you have to set the DELIVERY_STORE_BACKEND-setting in your settings.py like this:
DELIVERY_STORE_BACKEND = 'devilry.apps.core.deliverystore.FsDeliveryStore'
The FsDeliveryStore also require you to define where on the disk you wish to store your files in the DELIVERY_STORE_ROOT-setting like this:
DELIVERY_STORE_ROOT = '/path/to/root/directory/of/my/deliverystore'
To create your own DeliveryStore you have to implement DeliveryStoreInterface. A good example is FsDeliveryStore:
class FsDeliveryStore(DeliveryStoreInterface): """ Filesystem-based DeliveryStore suitable for production use. It stores files in a filesystem hierarcy with one directory for each Delivery, with the delivery-id as name. In each delivery-directory, the files are stored by FileMeta id. """ def __init__(self, root=None): """ :param root: The root-directory where files are stored. Defaults to the value of the ``DELIVERY_STORE_ROOT``-setting. """ self.root = root or settings.DELIVERY_STORE_ROOT def _get_dirpath(self, delivery_obj): return join(self.root, str(delivery_obj.pk)) def _get_filepath(self, filemeta_obj): return join(self._get_dirpath(filemeta_obj.delivery), str(filemeta_obj.pk)) def read_open(self, filemeta_obj): filepath = self._get_filepath(filemeta_obj) if not exists(filepath): raise FileNotFoundError(filemeta_obj) return open(filepath, 'rb') def write_open(self, filemeta_obj): dirpath = self._get_dirpath(filemeta_obj.delivery) if not exists(dirpath): makedirs(dirpath) return open(self._get_filepath(filemeta_obj), 'wb') def remove(self, filemeta_obj): filepath = self._get_filepath(filemeta_obj) if not exists(filepath): raise FileNotFoundError(filemeta_obj) remove(filepath) def exists(self, filemeta_obj): filepath = self._get_filepath(filemeta_obj) return exists(filepath)
We provide a mixing-class, devilry.apps.core.testhelpers.DeliveryStoreTestMixin, for you to extend when writing unit-tests for your DeliveryStore. Here is how we test FsDeliveryStore:
class TestFsDeliveryStore(DeliveryStoreTestMixin, TestCase): def setUp(self): self.root = mkdtemp() super(TestFsDeliveryStore, self).setUp() def get_storageobj(self): return FsDeliveryStore(self.root) def tearDown(self): rmtree(self.root)
Bases: devilry.apps.core.testhelper.TestHelper
Mixin-class that tests if devilry.core.deliverystore.DeliveryStoreInterface is implemented correctly.
You only need to override get_storageobj(), and maybe setUp() and tearDown(), but make sure you call super(..., self).setUp() if you override it.
You must mixin this class before django.test.TestCase like so:
class TestMyDeliveryStore(DeliveryStoreTestMixin, django.test.TestCase):
...
You might need to set the DeliveryStore manually if you need to handle deliveries in your own tests. Just set devilry.apps.core.FileMeta.deliveryStore like this:
from django.test import TestCase
from devilry.apps.core.models import FileMeta, Delivery
from devilry.apps.core.deliverystore import MemoryDeliveryStore
class MyTest(TestCase):
def test_something(self):
FileMeta.deliverystore = MemoryDeliveryStore()
delivery = Delivery.begin(assignmentgroup, user)
delivery.add_file('hello.txt', ['hello', 'world'])
delivery.finish()
The recommended DeliveryStore is devilry.apps.core.deliverystore.FsHierDeliveryStore.
It stores files in a filesystem hierarcy with one directory for each Delivery, with the delivery-id as name. In each delivery-directory, the files are stored by FileMeta id.
The delivery directories are stored in a hierarchy with two parent directories. The parent directories are numeric intervals. We have one top-level directory for each N in interval_size*interval_size*N. Within each toplevel directory, we have one subdirectory for each N in interval_size*N.
For interval_size of 1000, this will use the following hierarchy:
0/
0/
0/
1/
.
.
1/
1000/
2000/
.
.
2/
.
.
999/
1/
0/
1000000/
1000001/
.
.
1/
1001000/
1001001/
2/
.
.
999/
2/
.
.
999/
Bases: exceptions.Exception
Exception to be raised when the remove method of a DeliveryStore does not find the given file.
Bases: object
The interface all deliverystores must implement. All methods raise NotImplementedError.
Return a file-like object opened for reading.
The returned object must have close() and read() methods as defined by the documentation of the standard python file-class.
| Parameters: | filemeta_obj – A devilry.core.models.FileMeta-object. |
|---|
Return a file-like object opened for writing.
The returned object must have close() and write() methods as defined by the documentation of the standard python file-class.
| Parameters: | filemeta_obj – A devilry.core.models.FileMeta-object. |
|---|
Remove the file.
Note that this method is called before the filemeta_obj is removed. This means that the file might be removed, and the removal of the filemeta_obj can still fail. To prevent users from having to manually resolve such cases implementations should check if the file exists, and raise FileNotFoundError if it does not.
The calling function has to check for FileNotFoundError and handle any other error.
| Parameters: | filemeta_obj – A devilry.core.models.FileMeta-object. |
|---|
Bases: devilry.apps.core.deliverystore.DeliveryStoreInterface
Filesystem-based DeliveryStore suitable for production use.
It stores files in a filesystem hierarcy with one directory for each Delivery, with the delivery-id as name. In each delivery-directory, the files are stored by FileMeta id.
| Parameters: | root – The root-directory where files are stored. Defaults to the value of the DELIVERY_STORE_ROOT-setting. |
|---|
Bases: devilry.apps.core.deliverystore.FsDeliveryStore
Filesystem-based DeliveryStore suitable for production use with huge amounts of deliveries.
| Parameters: |
|
|---|
>>> fs = FsHierDeliveryStore('/stuff/', interval=1000)
>>> fs.get_path_from_deliveryid(deliveryid=2001000)
(2, 1)
>>> fs.get_path_from_deliveryid(deliveryid=1000)
(0, 1)
>>> fs.get_path_from_deliveryid(deliveryid=1005)
(0, 1)
>>> fs.get_path_from_deliveryid(deliveryid=2005)
(0, 2)
>>> fs.get_path_from_deliveryid(deliveryid=0)
(0, 0)
>>> fs.get_path_from_deliveryid(deliveryid=1)
(0, 0)
>>> fs.get_path_from_deliveryid(deliveryid=1000000)
(1, 0)
Bases: devilry.apps.core.deliverystore.DeliveryStoreInterface
Dbm DeliveryStore ONLY FOR TESTING.
| Parameters: |
|
|---|
Bases: devilry.apps.core.deliverystore.DeliveryStoreInterface
Memory-base DeliveryStore ONLY FOR TESTING.
This is only for testing, and it does not handle parallel access. Suitable for unittesting.