A Questionnaire for Assessing Immersive Websites

Web-immersion – i.e., a deep sense of cognitive and perceptual absorption engendered by the content and interactive features of a website - plays an important role in our modern, digital world. Yet, this topic has received little attention from Design and HCI scholars. The lack of tools to assess and measure immersion in the Web may severely limit our ability to understand the nature of such experience, thereby constraining future research in the area. To address this issue, we designed a questionnaire for assessing immersive websites and conducted a preliminary evaluation of it. In this article, we outline the questionnaire design and report on findings from a preliminary study conducted to analyze its reliability and validity. Moreover, we present results from Factor Analysis performed to investigate the dimensionality of the instrument. Finally, we conclude by discussing the implications of our findings, along with limitations of the study and future work.


INTRODUCTION
Three years after the start of the COVID-19 pandemic, a substantial part of our everyday life still takes place online. The Web has become the primary interface; the main entrance into such a new, digital 'reality'. As we live in a world where dichotomies such as 'close/distant' or 'virtual/real' have lost most of their meaning, our online interactions should be carefully (re-)designed. It has been long recognized that the focus of web design needs to shift from pure usability to a more experiential perspective (Thielsch et al., 2014). The quality of a website is indeed no longer determined by its utilitarian/functional attributes alone. On the contrary, users base their judgment of websites on experiential/hedonic values too (Huang, 2003). For example, attributes such as sensual novelty and aesthetic satisfaction proved to be strong drivers for first impression formation in website use (Kim, 2019). Moreover, it is observed that "experiential design" is an increasingly popular trend in web design (Susanto, 2020); suggesting that, now more than ever, an effective web experience should contemporarily affect users' sensations, emotions, and cognition (Susanto, 2020), and ultimately deliver an illusion of reality by transporting users into the digital 'space' (Gao & Li, 2019). In other words, we argue that in the current scenario surfing a website should generate a sense of immersion.
In this context, immersion is described as a deep sense of cognitive and perceptual absorption in the content and/or in interacting with a website (Scuri, 2017). Thus, an immersive website Scuri, S.; is much more than a 'digital document'; it's "a space to be explored, something to be experienced" (Emanuel et al., 2015).
Despite the extensive body of literature on immersion in Virtual Reality and game environments, the nature of such experience on the Web has received scant attention, with very few studies to date (Jennings, 2000;Coyle & Thorson, 2001;Emanuel et al., 2015;Scuri, 2017). Due to the growing ubiquity of web-based applications, web-immersion is emerging as a primary topic of interest for design research. Yet, the lack of tools to assess and measure immersion in the web may limit future research in this area. To address this issue, we designed and conducted a preliminary evaluation of a questionnaire for assessing immersive websites, which focuses on how multimedia cues and interactive features affect the experience of immersion. In this regard, it's worth mentioning that the present research focuses on desktopbased web browsing only. Such a decision was motivated by the fact that mobile user experience and desktop user experience are inherently different (Mendoza, 2013).
Specifically, it was decided to exclude the mobile web environment for two main reasons: • Characteristics of the device -"mobile devices are smaller than laptops with limited screen sizes, display resolution, storage capacity, and processing capabilities" (Wrede, 2018). For this reason, mobile-based interaction is less engaging compared to its desktop counterpart (Mendoza, 2013).
• Modes of interaction and mindset of users when browsing via mobile -desktop users are more focused on the screen and generally spend more time interacting with the website. On the contrary, mobile users normally have a 'bite-sized' experience (Mendoza, 2013), as most of the time the interaction takes place while the user is on the go. For this reason, immersion via mobile is not only harder to achieve but might also be undesired and, in some cases, even dangerous as it could divert their attention from the 'outside world'.
The contribution of this study is two-fold. First, the development of the scale in itself could shed light and expand our understanding of web-immersion, thus providing guidance for further research in the field. Second, developing a reliable scale, designed ad-hoc for the web, allows for a more accurate assessment of the qualities of an immersive website and their contribution to the overall experience.
The present article is structured as follows: section 1 provides an overview of relevant literature on immersion in general and in the areas of Human-Computer Interaction (HCI) and Web Design in particular. In section 2 we describe the methodology adopted to design and evaluate the questionnaire. Results for reliability, validity and factor analysis are presented in section 3. Main findings are summarized and discussed in section 4. We conclude by describing the implication of this study, along with limitations and future work.

BACKGROUND
The feeling of 'being surrounded' by a mediated reality, here defined as immersion, has been studied across several disciplines. The academic literature provides a wide array of conceptualizations and frameworks for its analysis. However, due to the diversity of viewpoints dealing with this experience, immersion is still a nebulous concept, even named differently depending on the field of study. In this section, we attempt to map out the main approaches to the study of such an experience. It's important to point out that our review isn't

Immersion in Communication, VR, and Game Research
The concept of immersion has received considerable attention from media and communication scholars since the early '90s, resulting in a rich body of literature. Among the many frameworks available for analyzing such an experience, the one most worth mentioning (as it represents the theoretical basis of many others) is probably the one developed by Steuer (1992). According to this model, the sense of 'being there' in a mediated environment -what Steuer refers to as 'telepresence' -is determined by the vividness and interactivity of the media; defined respectively as "the representational richness of a mediated environment […], that is, the way in which an environment presents information to the senses", and "the extent to which a user can participate in modifying the form and content of a mediated environment in real-time". As pointed out by Steuer himself, the main advantage of this bidimensional model is that it can be used to classify a variety of media. Several are indeed the empirical studies that found evidence of a strong relationship between the experience of immersion and these two factors -see Schuemie et al., (2001) for an extensive list of these studies.
Vividness and interactivity are also included in the framework elaborated by Lombard & Ditton (1997). In their article, the authors provide an in-depth analysis of the concept of 'presence' -i.e., the illusion that a mediated experience isn't mediated -with the goal of paving the way for more systematic research on the topic. By analyzing how the concept is addressed within different academic fields, they identify a list of variables that engender the experience of presence in media users. These variables are classified into three main categories: • Media form, which refers to the formal characteristics of the media -in terms of 'sensory richness' and 'vividness' -, and includes aspects such as number and consistency of sensory outputs, visual display characteristics (e.g., image quality), and interactivity (e.g., consistency between inputs and response).
• Media content: the quality of the content itself, regardless of its presentation (e.g., the extent to which content is perceived as realistic, authentic, or credible).
• Characteristics of the users: e.g., willingness to suspend disbelief or prior experience with the medium. • Reciprocity/Ease of response: the ability to facilitate real-time, reciprocal communication; • Synchronicity/Context awareness, which is "represented by just-in-time information presented to the user;" • Connectedness/Ubiquity/Pervasiveness: the use of ubiquitous or pervasive computing that allows users to "access multiple platforms with a continuous experience;" Scuri, S.; • User control/Personalization: the degree of control the user has over the interactive experience; • Navigability/Accessibility: the ability to access content and perform a task; • Entertainment/Sensibility: the ability of the media to "communicate or react to users with human-like style and personality;" • Sensory stimulations/Multimodality: "the degree to which an interface involves human senses".
Undoubtedly, the concept of 'immersion' is most commonly associated with Virtual Reality (VR). In this context, however, the term only refers to "what the technology delivers from an objective point of view" (Slater, 2003) -i.e., the extent to which a technology engages the user's senses. On the contrary, the subjective experience of 'being there' is called 'presence' and understood as "a human reaction to immersion" (Slater, 2003). Countless are the frameworks investigating presence in VR environments -a review of the main ones can be found in Schuemie et al., (2001). Despite the obvious differences between them, most frameworks present a common characteristic: they identify the objective features of the technology (i.e., immersion) as determinants of presence. An example of this is the model put forward by Sheridan (1996), which outlines three main determinants of presence: (i) fidelity of the multimodal displays (visual, acoustic, or tactile); (ii) ability to modify sensor position (e.g., change viewpoint); and (iii) ability to change the configuration of the environment (e.g., move objects).
Sheridan's framework has been further elaborated and extended by Slater & Wilbur (1997).
In their article, the authors speculate on the role of presence in virtual environments and ultimately outline five characteristics of immersive virtual environments that are supposed to increase presence. These are: • Inclusiveness: the extent to which physical reality is shut out; • Vividness: resolution, fidelity, and variety of the sensory stimuli; • Proprioceptive matching: the extent to which the participant's proprioceptive feedback about body movements matches the information generated on the displays; • Extensiveness: the range of sensory modalities accommodated by the system; • Plot: the extent to which there is a self-contained plot in which the participant can act and in which there is an autonomous response.
Finally, game research adopts a more holistic approach to immersion and looks at the player, rather than the game, as the primary focus of the investigation. As pointed out by Ermi & Mäyrä (2011), when it comes to immersive gameplay experiences, aspects like dimensions of the screen or high-quality audio are "by no means the only or even the most significant factor".
In their conceptualization of game immersion, the audio-visual execution of the game is indeed seen as only one of the possible dimensions of the experience, which consist of: • Sensory immersion: the extent to which the audio-visual execution of the game surrounds the player and causes him to be "entirely focused on the game world and its stimuli"; • Challenge-based immersion, experienced when there is a balance between challenges and abilities; • Imaginative immersion, experienced when the player is absorbed in the game world and/or emotionally connected with a game character.

Results from the preliminary study that Ermi & Mäyrä conducted to validate their model, not
only support the soundness of these three dimensions of immersion but also provide interesting insights into their individual significance, as well as into the way they may influence each other. For example, the authors argue that challenge-based immersion plays a particularly essential role as it promotes a flow-like experience -i.e., when the challenges are in balance with the players' skills and abilities. Moreover, factors related to the audio-visual design (sensory immersion) appear to contribute to generating imaginative immersion too.
Such a connection between immersion and players' senses, skills, and to some extent imagination, emerges in other works (Brown & Cairns, 2004;Jennett et al., 2008;Weibel et al., 2008;Weibel & Wissmath, 2011). Jennett et al., (2008), for example, conducted a study aimed at evaluating the effectiveness of subjective measures in assessing game immersion.
For that purpose, the authors elaborated an ad hoc questionnaire, whose items were derived from tools designed to measure (i) flow, (ii) presence (intended as vividness), and (iii) cognitive absorption (the player's attitude towards the game). Similarly, Weibel & Wissmath (2011), conducted a study to investigate the specific effect that flow and spatial presence (meant as vividness and realism of the game world) have on immersion in computer games and found empirical evidence that both are key concepts to explain such an experience.

Immersion in Web Design and HCI
Despite the extensive body of literature on users' immersion with media, the nature of such experience on the Web has received scant attention. To the best of our knowledge, existing Design and HCI research on the topic is very limited and still at an early stage.
In one of the few works dealing with this subject, Jennings (2000) draws on an extended concept of aesthetics (including perceptual, cognitive, and affective components) to propose a framework for designing immersive websites, which is nevertheless specific to a single web genre: e-commerce websites. In particular, Jennings outlines five principles that should be applied to e-commerce websites design: • Unity, which relates to the wholeness of the experience; the coherence and completeness of the content; • Attention or object directedness: the use of elements that bring about focus or a desire to proceed with the activity; • Active discovery: "the process of actively seeking answers or resolutions to cognitive challenges;" • Affect: the "emotional investment that helps create a personal link to an experience or activity;" • Intrinsic gratification: the "feeling of pleasure, reward and satisfaction" resulting from the experience. Scuri Coyle & Thorson (2001) adopt a more empirical approach to the subject matter. In their study, the authors examine the effects of interactivity and vividness in commercial websites. Using the model proposed by Steuer (1992), they manipulated the levels of vividness and interactivity of four websites and tested how those changes impact perceived telepresence and, more in general, users' attitudes towards the website. By doing so, they provided empirical evidence that perceived telepresence grows stronger as the levels of interactivity and vividness of a website increase.
In a recent study, Emanuel et al., (2015) investigate web-immersion by analyzing the so-called "rhetoric of interaction". Albeit acknowledging that multimedia cues and vividness are equally important in designing immersive websites, the authors postulate the crucial role of interaction (intended as a rhetorical element) in bringing users "into the action, the narrative, or the message". Based on the Aristotelian concepts of classical rhetoric, the authors attempt to explore how different interactive design solutions may influence users' experience with websites. Specifically, they seek to understand how interaction appeals to 'pathos' -i.e., the emotions of the audience. To do so, they analyze three very different websites and elaborate on the ways interactive features can be used to support the message. The third case study, for example, is the website of a company that sells lifejackets. Here, users are presented with a first-person-perspective video of a man, Julian, that accidentally falls off the boat on which he was traveling. The user is asked to help him stay on the surface by keeping scrolling or swiping.
After five minutes, regardless of the user's effort, the video ends with Julian drowning. As pointed out by Emanuel and colleagues, the use of a first-person perspective clearly helps the visitor connect with the character. However, it's the interactive features that make the experience 'ring true'. There is indeed a strong consistency between users' physical actions and character's virtual movements. Also, after five minutes of page-scrolling, the user will experience actual physical tiredness, which enhances the emotional connection with the character and his struggles. Although this study offers useful insights into how different interactive features could generate an immersive experience, it's still preliminary and doesn't provide an actual framework to follow. This aspect is acknowledged by the authors themselves, who stress the need for developing such a framework for the evaluation of immersive websites.
Given the heterogeneity of studies dealing with immersion, it isn't possible to provide a comprehensive overview of the subject (which would merit an article itself). However, a few important aspects do emerge from our review of the literature. First, formal and interactive features of the media are widely recognized as the key aspects of an immersive experience.
Second, there is no unique approach to the study of immersion. The way the experience is investigated (and even named) depends on the media context. Finally, the shortage of literature on web-immersion provides a stimulating research opportunity for the design community.

METHODOLOGY
This study explores users' perception of web-immersion. Specifically, we seek to develop a measurement instrument that identifies and assesses the 'immersive features' of websitesi.e., those design attributes that promote immersion. Due to the complexity of the subject matter, we adopted an exploratory approach to the study. In this section, the methodology used for the design and evaluation of the scale is described.

Items Generation
The questionnaire items were generated following a three-step process. First, we performed an extensive review of existing research and speculations about the factors that engender a sense of immersion in interactive media users. These sources were identified by searching the ACM digital library, Springer Link database, and Google Scholar. The search strings used included "immersion", "presence", "telepresence", "immersive experiences", and "immersive websites". For each article returned we checked the content of the abstracts. In order to be selected for full-text analysis, the article had to focus on one (or more) of the following subjects: • Conceptualization(s) of immersion; • Approach(es) or framework(s) for the analysis of immersion; • Measurement tools to assess perceived immersion; • Empirical studies to validate measurement tools and/or theoretical frameworks.
References of the selected articles were cross-checked to identify additional studies. A total of twenty-seven peer-reviewed articles were analyzed paragraph-by-paragraph, looking for the main characteristics associated with immersive media. Through this operation, we were able to extract a total of 318 'immersive features'. This initial list was reduced to 42 items by removing those that: • Depend on the characteristics of the hardware -e.g., screen resolution; • Require the use of devices other than a computer screen and a set of headphones; • Cannot be designed because depend on the users' characteristics (e.g., motivation and interest in the content).
The 42 remaining features were then analyzed using affinity diagrams during focus group sessions with experts in the field of Design and HCI. Those characteristics that were identified as having similar connotations were combined. Any disagreements were discussed until consensus was reached and a final list, consisting of 17 immersive features, was agreed upon.
Finally, we formulated a set of questions aimed at assessing users' perception of the 'immersive features' so identified. The resulting questionnaire consisted of 20 closed-ended questions, rated on a 7-point Likert scale.
To ensure clarity and accuracy of the items, the first version of the questionnaire was pilot tested on a small sample of researchers and Ph.D. students at the Interactive Technologies Institute (Madeira, Portugal). For this pilot, we adopted the think-aloud method, which has been widely used in assessing self-report questionnaires and is particularly useful to test the respondents' understanding of the items (Sudman et al., 1996). Based on pilot results, several items were reworded (or supplemented with comments in brackets) and one was removed.
The refined items and corresponding 'immersive features' can be found in the Appendix.

Questionnaire Evaluation
Reliability of the Immersive Features Questionnaire (IFQ) was tested using Cronbach's α. We then explored the dimensionality of the scale by using Exploratory Factor Analysis (EFA) via Principal Component Analysis (PCA). Finally, validity (discriminant and concurrent) was Scuri, S.; assessed by examining the relationship of the IFQ with two related but theoretically different constructs, which are widely recognized as the key components of immersion (Ermi & Mäyrä, 2007;Jennett et al., 2008;Weibel & Wissmath, 2011): Spatial Presence (SP) and Flow. SP is defined as the perceptual illusion of non-mediation (Lombard & Ditton, 1997), a sense of spatial immersion in a mediated environment (Weibel & Wissmath, 2011). The sense of presence is experienced when the mediated environment engages the user's senses (Witmer & Singer, 1994;Sheridan, 1992), thus, depends on the perceived quality and richness of the multimedia content. Flow (Csikszentmihalyi, 1990), on the contrary, is defined as the feeling of being "simultaneously cognitively efficient, motivated, and happy" (Moneta & Csikszentmihalyi, 1996). Such a construct refers to the mental state in which a person is fully immersed in what he/she is doing and can be used to assess and describe the quality of users' interaction with technology (Agarwal & Karahanna, 2000). For the purpose of this study, SP and Flow were measured using, respectively, the Presence Questionnaire (PQ) (Kim & Biocca, 1997) and the Flow Short Scale (FSS) (Rheinberg et al., 2002). We choose to use these scales as (i) they both have been used together to measure users' immersive tendency and found to be valid and reliable (Weibel & Wissmath, 2011);and (ii), like the IFQ, they're composed of closed-ended questions rated on a 7-point Likert scale.
In order not to limit our tool to a specific web genre, we performed three studies, using the questionnaire to evaluate different types of websites: • Study 1: a virtual walk (based on Google Street View technology); • Study 2: an interactive music video; • Study 3: a multimedia web-documentary about population decline in rural America.
Sample. Participants were design students and professionals, recruited through snowball sampling. They all signed an informed consent form and took part in the study voluntarily, without receiving any reward.
• Ninety-nine subjects (forty-one males and fifty-eight females) aged from 19 to 35 years (average age = 26.29 SD = 4.367) took part in study 1.
• Thirty-six participants (twenty males and sixteen females) aged from 19 to 35 years (average age = 27.86 SD = 4.051) took part in study 2.
• Thirty-four subjects (twenty males and fourteen females) aged from 19 to 35 years (average age = 27.76 SD = 4.164) took part in study 3.

Procedure.
After a brief introduction to the study, participants were given 15 minutes to freely browse the website (the study was conducted in an unsupervised setting to avoid any external influence) and then asked to complete an online questionnaire consisting of the FSS, PQ, and IFQ. Data Analysis. All statistical analyses were performed using IBM SPSS Statistics 26. Scuri, S.;

page 158
Reliability was tested using Cronbach's α. The IFQ achieved high internal consistency, with Cronbach's alpha coefficient of .86.
To further assess reliability, we examined the item-total correlation values. All items, except for item 5, appeared to be worthy of retention. The total-item correlation coefficient of item 5 was significantly below the threshold value of 0.30 (Cristobal et al., 2007). However, its removal would only increase the alpha to α = 0.87. Thus, further tests are needed to decide whether the item should be modified or removed.

Exploratory Factor Analysis
Prior to performing PCA, the suitability of data for factor analysis was assessed. Results from Bartlett's test of sphericity (χ2 = 1233.449, p<.001) and Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (KMO = .847) indicated that the data set was appropriate for factor analysis (Hutcheson & Sofroniou, 1999). The anti-image correlation was further checked to assess the sampling adequacy of individual items. All individual items were all well above the acceptable limit of 0.5 (Field, 2009) (values ranging from r =.650 to r =.912). The only exception was item 5 (r =.489), which only approached that limit.
Factor extraction was based on Kaiser's criterion, scree test, and interpretability. The EFA extracted five factors based on the Kaiser's criterion (eigenvalues ≥ 1), which accounted for more than 62% of the cumulative variance (Figure 1).

Figure 1. Total Variance Explained.
However, an inspection of the scree plot ( Figure 2) indicated that the majority of variance could be accounted for with a two-or three-factor solution. Therefore, solutions for two, three, and five factors were each examined using Oblimin rotation. The three-factor structure, which explained more than 50% of the variance, was preferred over the others because appeared to be the easiest to interpret and the most robust, with all primary factor loadings above .4 and few cross-loadings ( Figure 3).

page 160
Items loading on the first factor mostly describe quality and richness of the multimedia cues and, more in general, the extent to which content presentation 'surrounds' the user and affects his/her mood. Thus, we labeled this factor "Perceptual Immersion" (PI). The second factor consisted of six items describing aspects related to usability and perceived control. Hence, it was labeled "Operational Immersion" (OI). The four items belonging to the third factor describe the extent to which both multimedia content and modes of interactions are consistent and don't distract or interfere with the browsing experience. For this reason, we termed this factor "Cognitive Immersion" (CI).
PI investigates aspects like quality and richness of multisensory stimulation and degree of realism. It describes the ability of the media to 'surround' the users and make them feel 'there'; 'present' in the website. This dimension has several commonalities with factors that emerged in other studies. It includes most of the features that Lombard & Ditton (1997) categorize as "media form" variables. Also, it resembles the 'sensory immersion' advanced by Ermi & Mäyrä (2011). Most importantly, we assume that PI may contribute to Spatial Presence since the latter depends on the ability of a mediated environment to surround the users and engage their senses. Therefore, we expect to find a positive, high correlation between this dimension and the PQ.
OI is mostly associated with the interactive features of the websites, such as degree of control, responsiveness, usability, ease of use, etc., and seems to combine two of the dimensions of interactivity put forward by Tham (2018), namely navigability/accessibility and user control/personalization. It's also postulated that OI would show a correlation with FSS.
Several are indeed the items of the two scales that assess similar features, like perceived control and users' ability to predict the website's response to their actions.
Finally, the items belonging to the third form of immersion identified (CO) assess whether the website design interferes with the users' ability to focus on and interpret the content and/or achieve their goals. It presents some similarities with the notion of challenge-based immersion (Ermi & Mäyrä, 2011) and is therefore expected to show a correlation with the FSS.

Construct Validity
EFA was conducted on the items of IFQ and the two concurrent measures -PQ and FSS. For evidence of concurrent validity to emerge, the three subscales of our tool -PI, OI, and CIwould have to relate to PQ and FSS as hypothesized above. Evidence of discriminant validity will be established by finding a factor structure that corresponds to the different scales used in this study.
According to the literature, the items of both PQ (Kim & Biocca, 1997) and FSS (Engeser & Rheinberg, 2008) should be separated into two factors. Therefore, the EFA was conducted using seven factors for extraction with Oblimin rotation. Prior to performing the EFA, internal consistency (via Cronbach's α) of PQ (α = .859) and FSS (α = .859) was assessed, and the data set was examined for sample adequacy through Kaiser-Meyer-Olklin test (KMO = .883) and by Bartlett's sphericity test (χ2 = 3532.592, p = .000).
The seven factors extracted accounted for more than 62% of the variance. As shown in Figure   4, we found partial correspondence between the resulting structure coefficients and the three subscales. Scuri, S.; However, several items didn't have a simple structure, and a few of them loaded higher on other constructs. Such a result wasn't completely unexpected since the three scales have several semantically overlapping items and were assumed to be partially correlated. In particular, several items of the PI subscale had higher loadings on the Presence subscales than on their own. While two items of the OI subscale and one of the CI subscale loaded better on one dimension of Flow. To further assess concurrent validity, we analyzed the correlations (Spearman's rho) between PI, OI, CI, PQ, and FSS ( Figure 5). Results from the correlation test helped interpret the output of the FA. As suggested by Kline (1998) discriminant validity can be demonstrated if the correlations between factors aren't greater than 0.85. Although the correlations between the dimensions of web-immersion and the two concurrent constructs are high (between .781 and .660), they don't exceed such a limit.
It's also worth mentioning that, as hypothesized, OI and CI have a higher correlation with FSS than with PQ. While, according to our initial hypothesis, PI has a considerably high correlation with PQ (though it correlates significantly with FSS too).

DISCUSSION AND CONCLUSION
Web-immersion is becoming an increasingly relevant topic and certainly represents a fruitful area of investigation for the Design community. However, the limited literature available on the topic together with the lack of tools to assess and measure the 'immersive features' of websites, may limit future research in this area. To address this issue, we designed and conducted an initial evaluation of such a measurement tool.
Results from our study show that the IFQ has good internal consistency as measured by Cronbach's alpha (α = 0.86). Moreover, EFA suggests that web-immersion may be divided into three dimensions. The first one is represented by the following features: sensory depth and breadth, degree of realism, depth and field of regard of the visual cues, and ability to elicit proprioceptive responses. This dimension seems to describe quality and richness of the multimedia cues, as well as the extent to which content presentation 'surrounds' the user.
Hence, it was named Perceptual Immersion. The second dimension was labeled Operational Immersion since it's represented by items assessing usability and quality of the interaction, such as degree of control, immediacy of control, mode of control, and continuity. The third dimension consists of items that assess the extent to which a user can concentrate and perform the desired actions without being distracted by the attributes of the website; either those related to interactivity (e.g., mode of interaction) or those depending on vividness and content presentation (e.g., consistency of multimodal information). We named this dimension Cognitive Immersion. The dimensions of this three-factors structure show some similarities with findings from other studies, thereby lending partial support to the validity of the model, at least on a theoretical basis. Additional analyses were conducted to confirm discriminant and concurrent validity of the IFQ. EFA was performed to examine the relations between the three dimensions of web-immersion and other constructs commonly associated with the experience of immersion (flow and spatial presence). The factor structure resulting from the EFA had a moderate correspondence with the expected output. Although some items showed better loading on the concurrent validity measures, overall, all items had good loading onto Scuri, S.; their own factor. This result wasn't unforeseen as we expected to find positive correlations between the different measures.
Although encouraging, our findings don't provide clear evidence of construct validity. This might be due to several limitations of our study. The first limitation is represented by the moderately small sample size. Although there's no general rule for determining the sample size, it's commonly understood that larger samples yield more reliable results (MacCallum et al., 1999). Taking as a reference the scale proposed by Comrey & Lee (1992) for a rough evaluation of sample size adequacy in factor analysis, our sample (N = 169) would be considered between "poor" (N = 100) and "fair" (N = 200). Further limitations relate to the three scales used, which have several semantically overlapping items and, obviously, have never been tested together before. Finally, it's worth mentioning that almost all participants in our study were 'experts' -i.e., design students and professionals. Testing the scale with naïve users may lead to different results.
The study described here is preliminary in nature. Thus, further investigation is needed to support our initial findings. First of all, a confirmatory factor analysis should be conducted on a new set of data to validate the three-factor solution obtained in this study. Also, it would be interesting to explore whether other factors (e.g., the website genre) influence users' perception of one or more of the three dimensions of web-immersion.
Designing a valid measurement tool is a necessary step toward a better understanding of webimmersion. By establishing the different dimensions of web-immersion, our study provides a common framework for future research on the subject. Moreover, it outlines a set of distinguishable, measurable characteristics of immersive websites that could help inform design practitioners. To conclude, we should acknowledge that scale development and validation is an iterative process that never ends. This research represents just a first step in our quest to understand web-immersion and, perhaps, here lies its main contribution of this research.
I have been able to develop a mental map of the website.
Strongly disagree / Strongly agree Spatial awareness