Making the Visual Visible in Philosophy of Science

As data-intensive and computational science become increasingly established as the dominant mode of conducting scientific research, visualisations of data and of the outcomes of science become increasingly prominent in mediating knowledge in the scientific arena. This position piece advocates that more attention should be paid to the epistemological role of visualisations beyond their being a cognitive aid to understanding, but as playing a crucial role in the formation of evidence for scientific claims. The new generation of computational and informational visualisations and imaging techniques challenges the philosophy of science to re-think its position on three key distinctions: the qualitative/quantitative distinction, the subjective/objective distinction, and the causal/non-causal distinction.

Visualizations can expand processing capability by using the resources of the visual system directly.Or they can work indirectly by offloading work from cognition or reducing working memory requirements for a task by allowing the working memory to be external and visual.(Card, Mackinlay, and Shneiderman 1999, 16) Developers of visualising tools and techniques in engineering and computer science are sensitive to the need for an appropriate theory of vision to accompany the technology.1However, computational theories and metaphors of vision, of cognition, and of mind are widespread in the field, and talk of "high-bandwidth vision" capable of taking in information in the form of "bytes" which must be "processed" is very common.The effects of the conception of vision understood through the prism of computational metaphors in this domain and the ways in which this metaphorical framing becomes concretised in the design of these visualisations are not yet well understood.Two points especially demand our aention: the first is that cognitivist-computational theories tend to isolate vision as a psychological phenomenon, whereas for the epistemology of science, it is necessary to understand how vision works embedded in epistemic contexts, as playing a crucial role in the formation of evidence for claims.Second, cognitivist-computational theories tend to remain locked into

A. Carusi
Making the Visual Visible representationalist theories of vision and of visual artefacts, which generally do not do justice to the constructed, interactive, and material embodied way in which these work in scientific contexts.
The further irony is that even though the visual is everywhere in data-intensive science, it somehow remains in an epistemological blind spot.What follows is a series of suggestions for bringing the visual into view.
The philosophy of science needs to find ways of aending to the visual which treat it as having a problematic of its own, rather than being merely an appendage to the "real" scientific work which is carried out elsewhere.In 1996, in the Introduction to Picturing Knowledge, Baigrie noted that "the supposition persists that the pictures in science are psychological devices that serve as heuristic aids when reasoning breaks down" (xviii).Despite increasing aention to representation, of which van Fraassen's Scientific Representation: Paradoxes of Perspective (2008) is a notable example, aention to the visual as such is still scarce in philosophy of science today.Philosophers who have focused specifically on computational science oen mention how important visualisations are (for example, Winsberg 1999), but just how deeply they contribute to shaping the epistemology of computational science is still underestimated (though touched upon in several essays in Lenhard, Küppers, and Shinn 2006).2What I envisage in the context of this brief position piece is to set out what it might mean to take seriously the role of visualisations in the context of computationally intensive science; that is, to appreciate their crucial and irreducible role in establishing scientific claims.By this I mean a twofold study: on the one hand, a study of the ways in which visual evidence operates in computational science, and on the other, a study of the ways in which this mode of doing science plays a role in constituting the objects of science.
It is important to try to understand the different modes of visualising in science: image processing, scientific visualisation, and information visualisation each have different ways of connecting to data and information, and to the research object or domain under investigation.These are very different visual renderings.The distinctions are oen drawn along the lines of technologies and renderings we are already familiar with: for example, by distinguishing between those elements that are more akin to photographs which are seen to be operating in a causal relation with that which is depicted, and those which are more akin A. Carusi

Making the Visual Visible
to graphs operating in an informational mode with respect to what is depicted.The existence or not of a corresponding physical object is another touchstone of this distinction: an x-ray of a heart has such an object, whereas a graph of the rate of increase of taxation over the last 5 years does not.The term "image" is oen used of the first, and "information visualization" for the second, whereas "scientific visualisation" is used for those cases where there is a physical object which is simulated, and the visualisation is of the simulation rather than directly of the physical object.However, all of these ways of drawing these distinctions fail to capture the complexity of visual artifacts which are produced by multiple causal and computational criss-crossing lines.For example, the visualisation of a computational simulation not only has both image-like and graph-like elements, but combines them in ways that make it impossible to distinguish between them (Carusi and Hoel forthcoming).Similarly, the algorithms used in image processing mean that the resulting visual rendering is a hybrid of the causal and the computational.We need to find ways of re-thinking the different configurations of causal and informational or computational rather than trying to tackle the question by sorting them neatly into one box labelled "cause" and another labelled "everything else." The visual hybrids which are now endemic in science challenge us at deeper levels too.It is necessary to re-think or even give up three of the central distinctions in the epistemology of science.These are: the distinction between the quantitative and the qualitative, between the subjective and the objective, and finally, between the causal and the non-causal (understood variously as information, meaning, reason).These three distinctions are complexly inter-related in ways which this short position piece cannot fully analyse.I aim only to point towards possible research directions.
First, the distinction between the qualitative and the quantitative: I started out by talking about huge quantities of data being made tractable through qualitative visual renderings.alitative/quantitative reversals are the characteristic feature of digital visualisations of all kinds, occurring at different levels and in different ways.During the process of developing the technology, there will be continuous interplay between data in quantitative form, the algorithms for processing the data and producing the visualisation, and the qualitative visual evaluation of the progress of the algorithm formation.A similar process occurs in the actual use of visualisations in scientific contexts.The typical screen display, for example, is not only of a visual still or movie, but will also contain interfaces with quantitative parameter displays.The qualitative/quantitative distinction is also oen seen in terms of a continuous/discrete distinction: for example, the simulation of mathematical models involves moving back and forth between equations for continuous phenomena, the discretisation required for the computer simulation, and the visualisation which is again in continuous form.Thus the discrete is turned into the continuous, and the continuous back into the discrete in a constant Spontaneous Generations 6:1(2012) transformation, each cycle bringing about modifications and variations that inter-inform each other.There is not a straightforward one-way pipeline through the development of visualisation technology, nor through its use in a scientific seing.Instead, the process constantly doubles back onto itself, and there is an ongoing and reciprocal inflection of quantitative/discrete and qualitative/continuous, with each modifying the other.
Second, the distinction between the subjective and the objective: this oen follows in the footsteps of the qualitative/quantitative distinction.Despite the widespread reliance on the visual, distrust of the visual is still commonplace precisely because it is seen as subjective, and therefore variable.Visualisations are oen seen as temporary stopping points on the way towards the surer grounds of quantitatively couched answers, which have the objectivity that subjective perception is not seen to have, precisely because these answers appear to be invariable.For example, in information visualisation, the detection of paerns through the morass of huge quantities of data is carried out visually and qualitatively, by people (as opposed to by other computers).This is exactly what is pointed out as the huge advantage of visualisations in the cognitive-computational paradigm.However, the gold standard remains the numerical investigation of the paerns because it replaces subjectivity by objectivity.It promises a view from nowhere, undistorted by the inevitable cognitive limitations of individual people, or of groups of people.For this reason it appears to replace subjectivity by objectivity (Spencer 2011).However, following on the dissolution of the quantitative/qualitative distinction, it is less obvious that a clear distinction between the subjective and the objective can be upheld.This is closely inter-related with the third distinction to be considered.As Daston and Galison (2007) have shown, the alignment of the objective with the causal with the mechanistic, precisely because machines seem to exclude subjective bias, has been crucial to the formation of the epistemology of modern science.The advent of computational science throws this neat alignment further into disarray.
Third, the distinction between causal and non-causal: the second term in this distinction has been interpreted variously as relating to meaning, intention, or reason throughout the history of science, but in the context of computational science, it is oen interpreted as relating to information.For example, Barberousse, Franceschelli and Imbert draw a distinction between data E ("produced by physical interactions with measuring or detecting devices") and data A ("about a physical system") (2009,560).However, it is difficult to hold these two apart in information visualisation, in image processing, or in computational visualisation at the points of visualisation construction and interpretation.
The use of images in a human science, classics, is an interesting starting point in this investigation.The images of an ancient document in the form of

A. Carusi
Making the Visual Visible a wooden stylus tablet, the Frisian tablet, incorporated algorithms which were designed to mimic the motion made by papyrologists when they hold the tablet in the palm of their hands and "li it up at eye level against a light source and apply pitch-and-yaw motions to the artefact" (Tarte et al 2010, 139-40).There are many causal reasons relating to the depth of the incision in the tablet and the angle of the light as it is rocked back and forth which make this a fruitful practice for these papyrologists, but let us try to analyse what we have here: these causal interactions between hand, tablet, incisions, light, which are mathematically embodied in the algorithms for the image processing, are grounded in the concrete interpretative practice which made them salient (among all the other causal interactions possible in that context).Even in the palm of the hand, there is an interplay of the causal and the intentional-in both the sense of intendings (purposeful action) and of meanings, that is the targets of the interpretive practice.Embodied in the algorithm for image processing, there is a hybridity of causal factors (the way in which the algorithm organises shapes and contours in the image) and intentional/informational factors.The resultant images that are viewed for further interpretation are a hybrid of causal and non-causal factors, since in observing and inspecting them, there is no way of holding apart, in vision, what has been fused through material practice and technology, and the feedback and feedforward loops between them.Indeed, the image is in its turn interacted with materially, as the papyrologists trace with their fingers and through gestures the incisions made visible through the image processing as they engage with the images for further interpretations (de La Flor et al. 2010).This amalgam of the causal and the non-causal is not limited to the human sciences, but only more likely to be reflected upon there.The question is to understand how it operates across the sciences. 3nother example is the computational mesh that is used to run simulations and generate visualisations in simulations of all kinds.As applied to the investigation of cardiac electrophysiology, these meshes combine the results of histological images, algorithmic image processing techniques, and numerical simulation techniques, which in their turn are designed with a view to the mathematical models that must be solved through the simulation, and the experimental data that parameterise the models (described in detail in Carusi, Burrage and Rodriguez 2012).In these meshes and the resultant simulations and visualisations, the causal and the informational feed into one another at every turn, and it is this inter-relation that demands our aention.There is a real question regarding whether they can be picked apart in anything but the most artificial fashion.If they cannot be, we cannot judge how much their separate contributions can be depended upon in science, how much epistemic warrant they separately afford.Let us rather turn to understanding how they are inter-related.
A further task is to deepen the understanding of information in computational scientific seings.Information can be understood in a variety of different ways (Floridi 2008).The notion of information always brings in a relational element since it implies not only that there is something about which the information is, but also something or someone for which or for whom it is information.A common distinction is between data and information, where information is understood as data made meaningful, already organised and arranged in some way.On this conception, information always implies a point of view, an interest, a question in terms of which data is made meaningful.For something to be information, there must be an "appreciator" of the information.Shannon's theory of information in mathematical terms, which has informed so much of computer science, places this "for whom" in the shadows (Shannon and Weaver 1949); ordinary talk of information will oen aribute this "appreciation" to the computer ("the computer thinks that this is a contour, " "the computer bunches those things together, " and so on), and the aribution of a form of agency to computational entities is the topic of heated debate in the philosophy of information. 4Without entering into this debate, it is interesting to note that the understanding of the non-causal as information in this more relational sense harks to a more full-blown agency.For example, with computer programmes that are not limited to processing information, but operate on it in different ways, some of which may involve the ability to "make choices" or "take decisions" or "act" autonomously.It is in this sense that they are seen as agents, or agent-like.Agency has been jealously guarded as belonging exclusively to humans or higher beings, so the idea of extending it to computational entities is indeed radical.But it is not radical in the conception of agency that is put forward, since it still ties agency to discretely existing entities, be they computers or humans, whose agency results from acting autonomously (and therefore not mechanistically).A rather more interesting view of agency in the development, use, and circulation of digital and computational visualisations is offered by considering the subjective as not "housed" in humans or groups of humans, nor in computers or clusters of parallel computers, but as distributed across an epistemic-technological-social system.What might be treated as information is not a maer for any form of agency alone since what the computer might count as information cannot but be formed by what counts to its user as information, and how it processes that