Space-time possibly fractal dimensions in arts and books

I was reading a collection of stories from John Barrow (100 things..., 2014) and among maths and arts stories there was one (#58) about the dimensionality in space and time of artistic production. He argues about the fact that basic arts can be mapped to spatial dimensionality in 1,2 and three dimensions from lines to sculpture. And then it time dimension can be easily added to each of them as SNT.

In this short post I am providing some other examples and variation of this idea and making an example of connection between this spatial dimensionality and use in Machine Learning.





His examples are:

	N=1	N=2	N=3
T=0	line	picture	sculpture
T=1	music	cinema	theater

To the case of S3T1 we can add virtual reality dynamic sculpture (e.g. Google Tilt Brush)

Google TiltBrush 

Non-linearity and scale in space and time bring to different types of artistic schemes in which each dimension is explored at different rates. Emptiness in a sculpture, or amount of detail in a painting are such examples. Non-linearity in time is equivalently important, and even 4' 33" by John Cage is part of this scheme of pacing.

Things get more interesting when we consider fractal dimensions, first in space as in fractal lines and surfaces, capable of filling their upper dimension. And then, finally, Barrow highlights the possibility of fractal time citing the idea of a book whose dynamics is altered by the decision of the reader each time differently. This corresponds exactly to the scheme of game books mentioned in my previous post, in which the story can follow many different, intersecting paths based on reader intervention. This means that the time dimension is fractally pushed toward the second time dimension. A modern variation of the game books are the Lone Wolf video games (video) or the less demanding Telltale games in which the user has a fixed number of actions that are mixed with cinematics videos.

Images from the Lone Wolf game that keeps the style of the original: combats in 3D,
pure reading as text

Due to the nature of these books we have two time axis to be considered: one axis correspond to the flow of time in the story, that has variations in extent and articulation as shown in the graphs, the other axis is the real-time of the reader that due to the number of paths taken is going to read more, think about decisions or throw dices for combats. It would be interesting to measure these two extents by means of textual and content analysis.

I would like also to mentions two possible variations to the scheme described by Barrow in his piece. The first is the aspect of multiplicity, that is the exploration of the same entity with variation, think about the case of Rouen Cathedral by Monet, Andy Warhol paintings, or, looking at the time dimension, at movies with the repetition of the same story multiple times (e.g. Source Code, X-File "Monday" episode or more recently the Westworld series). The Edge of Tomorrow movie has a variation of such pattern with its repeating timeline extending at every iteration. Multiplicity in time could be represented  as k SN T or with fractal time dimensionality SN Td .

The other possible variation is projection that is the reduction of dimensionality as happens with shadows. This is the case of sculpture with meaningful shadows (e.g. Shadow Scultpures), or the Hinton Tesseract that's the projection in 3D of a 4D hypercube (mentioned in #77).

Tesseract (Thingverse)

Shadow Sculpture by Noble and Webster

The reverse is also possible with (up)projection. Many pictures have a strong depth perception, or we can mention the extreme case of Metanamorphs that are drawing on a pavement for which exist a single point of view with high 3D impression. A space filling curve can be also considered a way for (up)projecting from 1D to 2D, e.g. for visualizing 1D data in a space efficient way. I have used it in the past for visualizing in 2D the histogram of patent classes.

Peano space filling of 1D histogram in 2D (Ruffaldi 2010)

Projection can be easily applied to time when creating a filmstrip from a movie (that is from S2T1 to S2T0) or more interestingly chronophotographs (that is from S2T1 to S2Tk).

Chronophotograph by Xavi Bou

(Up)projection in time-domain is the creation of an animation from image sequences with the original Phenakistiscope. So what we can expect from (up)projecting from T to fractal time TK?

This interpretation of artistic or visual outcomes can is implicitly employed in Machine Learning research when we need to map a series of data input (in space or time, spanning from a single picture to a moving pointcloud) into a meaningful input for a learning system. This is the case for example of expressing time series in a projected way as done with windowing, rather then as pure time (RNN), and more exotically expressing timeseries as 2D images for exploiting convolution.

Updated 30th May: TellTale
Updated 4th Jun: added reference to Westworld as multiplicity