Neos has some powerful features under the hood which makes management of media files and documents easy. This post is the first part of a series about asset management in Neos CMS and Flow and explains the base layer of it all, the Flow Resource Management.

Can you tell the difference between Flow Resources, Neos Media Assets and assets managed by Neos CMS? Let's get these things sorted out first, before we start looking at the brand new cloud features in a later part of this series.

Flow Resource Management

If you create applications with Flow Framework, you'll get in touch with the Flow resource management. When we were thinking about the base functionality of Flow many years ago, I created an abstraction layer for files: instead of dealing with filenames and paths directly, developers work with objects (you'll see, later on, which benefit that has on cloud environments). But no matter if you are working on a Flow application or use Neos CMS as a site integrator, the foundation of it all is Flow's resource management.

Let's start defining a few terms which are frequently used in Flow:

Resource

A Resource is the "atom", the smallest part of an asset; it is the representation of the binary data which is usually stored as a file. That means, a Resource is the object which directly represents the binary data you or your users upload or import. There's no metadata stored in Resource objects, apart from the bare minimum (filename, type, modification date and size).

It is best practice to not share Resource objects between multiple parents. That means: if you have a "Conference" object which has a "logo" property (which is a Resource object), you should not re-use that Resource object for a second Conference instance. Just copy, re-import or re-upload it. Flow will make sure that no storage space is wasted and stores the binary data just once, no matter how many Resource objects are using it.

Static vs. Persistent

Static Resources are deployed as part of the code base. They never change during runtime. A Persistent Resource is data uploaded, created or imported by the application. Each resource is registered in the database.

Storage

The binary data of a resource is stored in a low-level storage. Resources of a storage are not publicly available and only programmatically accessible by your application.

Publishing Target

Resources are published to Publishing Targets. The Publishing Target provides a public URL to a single or multiple Resources.

Resource Collections

Each Resource belongs to one Collection. A Collection defines the Storage and Publishing Target to use for the contained Resources.

Alternative Storages and Publishing Targets

If you are like us and prefer running Neos in a scalable Cloud environment, you don't want to store uploaded Resources in a file system. When an application requires a persistent file system, scaling – and in general, using containers – can become much more of a hassle. Fortunately there are alternatives, and both Neos and Flow are playing really well without a persistent hard disk under their feet.

There are two plugins from Flownative you can use as an almost drop-in replacement for the standard storages and targets:

• flownative/aws-s3 provides a storage and target for Amazon S3 (and compatible solutions, like Minio!)
• flownative/google-cloudstorage provides the same for Google Cloud Storage

Low Level Tools for Resource Management

If your data is corrupt, or you just want to check if everything is in order, there are a couple of low level tools which can help you cleaning up your application.

First of all, ./flow resource:clean will check, which resource object should exist, by going through the entries stored in the database. The command will then compare the list of resources with the binary data actually stored in the storage – and if any of the resources are missing, will suggest to remove the stale entries from the database. Depending on if there are additional dependencies on these Resource objects, cleaning up may or may not run through without errors.

In every case, when some data is corrupt or missing, you should look further into why that happened in the first place. Don't rely on clean-up tools but rather make sure that these things can't happen.

Then, there's another command called ./flow resource:copy which allows you to copy binary data from one collection (ie. the storage of a collection) to another. This command is quite handy when you're migration to or from a remote storage, such as Google Cloud Storage. Check the Flownative S3 and GCS plugin documentation for some examples using this command.

Next: Images, variants, thumbnails and animated GIFs

Now you have a basic impression of what goes on at the lowest level. In the next part of this series we'll look at Neos.Media, the package which provides powerful and easy-to-use ways to deal with images, documents, thumbnails and more.

Do you like the approach of storing assets in an infinitely-scalable cloud storage but spare the effort to set everything up? Then Flownative Beach, our cloud-platform, might be something for you. It was designed for Neos and Flow from the ground and takes all the cloudy hassle from you. Learn more about Beach