Introduction
Irritable bowel syndrome (IBS), as one of the most common functional gastrointestinal disorders, seriously affects patients’ quality of life, with a prevalence rate of approximately 7.4% in China.1 IBS can be categorised into four subtypes: IBS with predominant diarrhoea (IBS-D), IBS with predominant constipation (IBS-C), IBS with mixed bowel habits (IBS-M) and IBS unclassified (IBS-U).2 A global meta-analysis revealed that the prevalence rates of IBS-D, IBS-C, IBS-M and IBS-U were 31.5%, 29.3%, 26.4% and 11.9%, respectively, among patients with IBS.1 A survey conducted in Guangdong Province, China, found that 19% of individuals meeting Manning criteria and 22% of those meeting Rome II criteria in the community population suffered from IBS.3 The low consultation rate can be largely attributed to the lack of awareness regarding IBS, thus emphasising the need for further promotion of health knowledge dissemination.4 Social media platforms are currently recognised as one of the primary channels through which people obtain health information.5 TikTok, Kwai and BiliBili are three widely popular video-sharing platforms in China.6 Patients can easily search for relevant disease-related information using keywords. Although social media provide numerous opportunities to acquire health knowledge, they also possess certain limitations. Studies have investigated disease-related short videos on several video-sharing platforms, such as TikTok,7–9 and have shown that many of the videos are not reliable; some even disseminate misinformation.10–12 For instance, videos about IBS on TikTok often recommend supplements that have not been approved by the U.S. Food and Drug Administration.13 This is primarily due to the absence of professional content review before these videos are uploaded onto these platforms. The presence of misleading and false information in health-related videos may increase the risk of patients making poor decisions. Therefore, the information conveyed by health-related videos must be reliable and of quality. This cross-sectional study aimed to assess the quality and reliability of IBS-related information on the most popular Chinese short-video platforms: TikTok, Kwai and BiliBili.
Methods
Search strategy and data collection
A new account was created and logged into on each video platform, using ‘irritable bowel syndrome’ as the Chinese keyword. On 1 November 2023, searches were conducted on TikTok, Kwai and BiliBili. The top 100 videos recommended by each platform’s ranking process were retrieved for analysis.14 Duplicate, irrelevant and commercial ads are excluded from the data set. Two authors (Yunfeng Huang and Jianfang Rong) jointly identified, recorded and scored the final videos included in the analysis, and the final score was given after full discussion with the third author (Shu Xu).
Video characteristics
Video sources we categorised into professional institutions, non-professional institutions, health professionals, science communicators and general users based on the self-introduction of each video uploaders. For example, doctors are defined as ‘health professionals’; medical institutions or medical science popularisation platforms are defined as ‘professional institutions’. Self-described ‘science bloggers’ are defined as ‘science communicators’. Television stations and online media are defined as ‘non-professional institutions’. The rest are defined as ‘general users’. Basic information was recorded, including the identity and type of uploader, number of days since published, length and content of the video, number of likes, comments and favourites. Elements of the video were also recorded, including the presence of people, use of background music, emoji, animations and subtitles.15
Assessment of content and quality of videos
The Journal of the American Medical Association (JAMA) criteria employ a set of four questions that have been used to assess the transparency and reliability of Internet information. Each video was assigned a score ranging from 0 (lowest) to 4 (highest).16 17 The quality of information within a video was evaluated using the Global Quality Scale (GQS), with scores ranging from 1 (lowest) to 5 (highest). A higher score indicates better quality.18 19 To demonstrate the reliability of the video content, a modified DISCERN questionnaire with five questions was used. A higher score signifies greater reliability.17 20 The video content was analysed using the six questions proposed by Goobie et al (definition of the disease, its signs or symptoms, risk factors, diagnosis, treatment and complications).21 Each question received a score of 0 (unresolved), 1 (partially resolved) or 2 (fully resolved).15 The scoring details of each scale can be found in the Evaluation Criteria (online supplemental material).
Statistical analysis
IBM SPSS Statistics version 23 (IBM Corp., Armonk, NY, USA) and Prism GraphPad version 9.5.1 (GraphPad Software Inc., La Jolla, CA) were used for data analysis. Descriptive statistics include median and IQR. Intraclass correlation coefficients were used to calculate two-rater agreement. The average agreement by intraclass correlation coefficient was 0.823 for JAMA rating, 0.851 for GQS rating, 0.830 for modified DISCERN rating, and 0.892 for Goobie et al’s rating, indicating that the rating has good consistency.15 22 23 In this study, the Kruskal–Wallis H test with Dunn’s multiple comparisons test was applied to compare multiple groups with non-normally distributed quantitative variables. Spearman correlation analysis was conducted to evaluate the relationship between video characteristics. Univariable and multivariable ordinal logistic regression analyses were conducted as used to analyse the influencing factors of the scores. Pearson correlation analysis was used to evaluate the relationship between video characteristics and the GQS scores and modified DISCERN scores. A P-value less than 0.05 was considered to be statistically significant.
Results
Video characteristics
The top 100 IBS-related short videos were reviewed on TikTok, Kwai and BiliBili, a total of 244 videos were selected for inclusion in this study based on predefined criteria (figure 1). Most of the videos have subtitles (95.5%, 233/244) and people appearing (83.2%, 203/244). Videos using background music and animation accounted for 49.6% (121/244) and 24.2% (59/244), respectively. Only 23.4% (57/244) of videos used emoji. Health professionals uploaded the most videos, accounting for 68% (166/244), followed by science communicators (10.2%, 25/244), general users (8.6%, 21/244), and professional institutions (7%, 17/244). Non-professional institutions uploaded the fewest videos (6.1%, 15/244) (table 1).
Video characteristics
Flowchart for the filtering of irritable bowel syndrome-related short videos for analysis.
Comparison of different video platforms
On TikTok, professionals uploaded the most videos, accounting for 75.6% (62/82), followed by professional organisations (7.3%, 6/82), general users (6.1%, 5/82), and science communicators (6.1%, 5/82). Similarly, professionals accounted for the majority on Kwai with a percentage of 72.1% (62/86), while general users contributed to around 11.6% (10/86) and science communicators accounted for approximately 7.0% (6/86). On BiliBili, professionals uploaded the most videos at a rate of 55.3% (42/76), followed by popular science practitioners contributing to about 19.7% (15/76) and professional organisations accounting for approximately 9.2% (7/76) (figure 2, online supplemental table 1).
Percentage of irritable bowel syndrome-related videos from different sources on TikTok, Kwai and Bilibili. (A) Sources of TikTok videos; (B) Sources of Kwai videos (C); Sources of BiliBili videos.
Compared with the other two platforms, TikTok videos have more people presence and emojis, Kwai videos most often used background music, while BiliBili videos used animations more (online supplemental file 2).
Videos on BiliBili were longer than those on TikTok and Kwai, but the number of likes, comments and collects was higher on TikTok than those on Kwai and BiliBili (p<0.001).
To assess the transparency and reliability of the videos, the JAMA median scores for TikTok, Kwai and BiliBili videos were 3 (IQR 2–3), 3 (IQR 2–3) and 2 (IQR 2–3) respectively. TikTok and Kwai had significantly higher scores compared with BiliBili (p<0.01 and p<0.05, respectively). In assessing video quality using GQS scores, the median scores for TikTok, Kwai and BiliBili were 3 (IQR 2–4), 3 (IQR 2–4) and 3 (IQR 3–4), respectively. BiliBili’s score was higher than Kwai’s (p<0.05). Using the modified DISCERN score to assess the availability and reliability of the videos, the median scores for TikTok, Kwai and BiliBili were 3 (IQR 2.75–3), 3 (IQR 3–3) and 3 (IQR 2–4) respectively. The difference in modified DISCERN scores among the three platforms was not significant (p=0.364) (table 2, figure 3, online supplemental table 3).
Comparison of different short-video platforms
Scores for JAMA, GQS, modified DISCERN and Goobie’s questions on the three video platforms. (A) JAMA scores, (B) GQS scores, (C) modified DISCERN scores and (D) Goobie’s questions’ scores . *p<0.05, **p<0.01, ns: not significant at p<0.05.
Analysis of video content
To further investigate the content of IBS-related videos, six questions proposed by Goobie et al were used to analyse the videos. Most of the videos provided a comprehensive description of symptoms and signs, whereas videos describing complications were the least common (online supplemental figures 1 and 2). The median scores for TikTok, Kwai and BiliBili were 7 (IQR 5.75–8), 6 (IQR 5–8) and 8 (IQR 6–9) respectively (table 2). Among the 17 videos that addressed all six questions, TikTok had seven videos, whereas Kwai and BiliBili had five videos each.
Differentiating the videos based on their sources, the median scores for science communicators, health professionals, professional institutions, non-professional institutions and general users were 8 (IQR 6–8.5), 7 (IQR 6–9), 7 (IQR 4.5–8.5), 6 (IQR 5–8) and 3 (IQR 2–4.5), respectively (online supplemental table 7). The score of general users was significantly lower than that of science communicators, health professionals and professional institutions (p<0.0001, p<0.0001 and p=0.0005, respectively) (figure 4).
Scores for JAMA, GQS, Modified DISCERN and Goobie’s questions for the five video sources. (A) JAMA scores, (B) GQS scores for the five video sources, (C) modified DISCERN scores and (D) Goobie et al.’s questions’ scores for the five video sources. *p<0.05, **p<0.01, ***p<0.001, ****p<0.0001.
Assessment of different video sources
Videos uploaded by science communicators lasted the longest, whereas videos uploaded by health professionals had the highest number of likes, comments and favourites. The scores were further compared among different video sources (table 3, online supplemental table 4). The highest median score in JAMA was obtained by health professionals with a score of 3 (IQR 2–3), followed by professional institutions with a score of 2 (IQR 2–3) and science communicators with a score of 2 (1-2) (table 3). These scores were significantly higher than those of general users, which had a median score of 1 (IQR 0–1; p<0.0001, p<0.001 and p<0.05, respectively).
Comparison of different video sources
For GQS scores, the highest median score was achieved by science communicators with a score of 4 (IQR 3–4), followed by health professionals with 3 (IQR 3–4) and professional institutions with 3 (IQR 2–4). These scores were significantly higher than those of general users, which had a median score of 2 (IQR 1–2; p<0.0001, p<0.0001, and p<0.01, respectively).
Regarding modified DISCERN scores, the highest median score was attained by science communicators with a score of 3 (IQR 3–4), followed by health professionals with 3 (IQR 3–3) and professional institutions with 3 (IQR 2–3). These scores were significantly higher than those of general users, which had a median score of 2 (IQR 1–2) (p<0.0001, p<0.0001, and p<0.05, respectively). The scores were not significantly different between non-professional institutions and general users (p>0.05) (figure 4).
Influencing factors for JAMA and GQS scores
The above results revealed significant differences in JAMA and GQS scores among the three platforms. Owing to the significant differences in video duration and the proportion of video sources on each platform, univariable and multivariable analyses revealed that video source (adjusted OR (aOR) 19.849, 95% CI 9.594 to 41.063; p<0.001) was the influencing factor of JAMA scores (online supplemental table 5). Video duration (aOR 8.104, 95% CI 4.548 to 14.441; p<0.001) and video source (aOR 5.132, 95% CI 2.855 to 9.225; p<0.001) were the influencing factors of GQS scores (online supplemental table 6).
Correlation analysis
Spearman correlation analysis revealed that the following variables were correlated positively: days since published and duration (r=0.31, p<0.001), duration and comments (r=0.18, p=0.006), duration and collections (r=0.28, p<0.001) (r=0.88, p<0.001; online supplemental table 8).
Days since published (r=0.19, p=0.003) and duration (r=0.27, p<0.001) are positively correlated with GQS scores, whereas (r=0.18, p=0.004), comments (r=0.21, p=0.001) and collections (r=0.21, p=0.001) are positively correlated with modified DISCERN scores (table 4).
Pearson correlation analysis between video characteristics and the GQS and modified DISCERN scores
Discussion
IBS is the most common gastrointestinal disorder of the brain–gut axis,24 characterised by abdominal pain, bloating, constipation or diarrhoea.25 Patients with IBS often accompanied by anxiety and depression,26 and psychiatric symptoms are positively correlated with the severity and frequency of intestinal symptoms.27 It primarily affects young and middle-aged individuals and is more prevalent in women. It significantly affects the lives, work and psychological well-being of patients.28 Research has reported that the quality of life in patients with IBS is even lower than that of patients with conditions such as diabetes, gastro-oesophageal reflux disease and end-stage renal disease requiring dialysis.29 However, not many patients actively seek medical treatment. The worldwide consultation rate for IBS has been reported to be around 40%,30 and it is only approximately 20% in China.3
With the widespread use of smartphones, people increasingly prioritise obtaining health information from the internet.31 Short videos have gained popularity among users owing to their quick and vivid nature.14 With more than 500 million users in China, TikTok, Kwai and BiliBili play an important role in the dissemination of information.20 These platforms can be effective tools for disseminating health information; TikTok even became a platform for the Ministry of Health to provide authoritative information and correct misinformation about COVID-19 during the COVID-19 pandemic.32 33 Therefore, it is important to ensure the reliability and accuracy of information in health-related videos. Previous studies have evaluated the quality of some disease-related short videos, such as gastric cancer,8 34 liver cancer9 and Helicobacter pylori infection.7 35 However, to date, no study has assessed the quality of information related to IBS on video-sharing platforms. In this study, 244 IBS-related short videos were reviewed on TikTok, Kwai and BiliBili. TikTok was the most popular, and BiliBili has the longest video. TikTok has the highest proportion of health professionals, whereas Kwai has the highest proportion of general users.
The JAMA scores of TikTok and Kwai videos were higher, which may be due to the stricter review of health information videos on TikTok and Kwai. The GQS scores for videos on Kwai were lower than those of TikTok and BiliBili, which may be due to the higher percentage of videos uploaded by general users on Kwai. This indicates that the reliability and quality of IBS-related health information on TikTok are more satisfactory, aligning with the findings of previous research.14 36 There was no significant difference in the modified DISCERN scores for the three platforms. The differences in video quality among platforms may be attributed to the characteristics of their user groups: BiliBili has a majority of young viewers who prefer high-quality, longer videos. Kwai is very popular among users in small cities; however, it lacks an authentication mechanism. Conversely, TikTok has attracted a wide range of age groups and has the largest user base. Moreover, TikTok requires platform certification before posting health-related content, making it highly reliable.37
In addition, videos made by professional individuals or science communicators with disease knowledge-focused content were of higher quality and more reliable. In general, health professionals uploaded the most videos, with higher quality and popularity. The videos uploaded by science communicators had the highest quality, which may be related to their longest video length, which is consistent with previous research conclusions.7 Their lower JAMA scores may be due to the fewer videos indicating the source of information citation. Professional institutions were comparable with health professionals in terms of video quality and content integrity. Most videos uploaded by general users did not indicate the source of information citation or had a large amount of missing content, with poor quality, reliability and content integrity. However, as patients with IBS are greatly affected by mental health,38 some videos of personal experience released by general users can cause resonance among many patients. Therefore, although the quality of videos released by general users is not high, the number of likes is second only to that of health professionals and science communicators. In terms of video content integrity, BiliBili seems to score higher than TikTok and Kwai. The score of science communicators was the highest, followed by health professionals and professional institutions. Videos of science communicators were longer. Approximately 93% of the videos are missing at least one content, so the overall content integrity score is not high. In addition, some videos were of low quality because the authors recommended drugs or supplements that were not approved by the FDA, such as traditional Chinese medicine.
Univariable and multivariable logistic regression analyses showed that platform and video source were the common influencing factors of the JAMA score, whereas video quality was mainly affected by video duration and source. Correlation analysis shows that days since publication and duration are positively correlated with video quality. This implies that high-quality videos tend to be longer, whereas newly uploaded videos generally exhibit lower quality. Previous studies have indicated that useful videos are longer than misleading ones,39 possibly reflecting a growing preference among viewers for quick answers over lengthy, high-quality content.40 No obvious correlation exists between the popularity and quality of videos, which indicates that the viewers’ ability to recognise high-quality videos needs to be improved. Likes, comments and shares are positively correlated with modified DISCERN scores. This indicates that the more popular videos tend to be more reliable, which may be due to the stricter requirements of various platforms on the qualifications of medical video uploaders in recent years, and the platforms tend to recommend videos released by doctors or medical institutions, which is consistent with previous studies that not only videos with high likes are recommended to the top of the list, but also videos of some doctors41.
In general, the videos analysed in this study were evaluated using JAMA, GQS, modified DISCERN and Goobie et al.’s six questions, and all obtained medium scores of 2 (IQR 2–3), 3 (IQR 2–4), 3 (IQR 3–3) and 7 (IQR 5–8), respectively. These findings suggest that the overall quality of IBS-related short videos is poor. Health professionals and science communicators should take advantage of their high popularity to produce more high-quality short videos so that more people can understand IBS and receive early diagnosis and treatment. Video platform operators and online regulators should monitor health-related video information to prevent misleading and fraudulent videos from spreading online.42 The public must improve their judgement ability, select health-related videos uploaded by reliable sources and seek timely medical advice from specialists when necessary.43
Strengths and limitations
To our knowledge, this is the first study to analyse the reliability and quality of IBS-related short videos across the three major video-sharing platforms in China. These platforms cover the majority of short video users in China, evaluating these three video platforms can obtain more comprehensive and reliable findings. For video evaluation, three different tools were chosen to assess the transparency, reliability and quality of the video and the six questions proposed by Goobie et al to analyse the content of the videos, so as to analyse the video information from multiple dimensions. The correlation among the days since publication, duration, likes, comments, collections, as well as video quality and reliability, was further analysed, and the days since publication, duration and video quality were positively correlated, whereas likes, comments and collections were positively associated with video reliability.
However, this study has certain limitations. First, search results can be influenced by variations in search outcomes based on user behaviour. Second, since video uploaders can update or delete videos at any time, the videos searched for may be different at different times. Besides, there may be limitations in the use of JAMA and DISCERN tools to assess video content since these tools were primarily designed for evaluating website information rather than short videos. Furthermore, some video source groups had relatively small sample sizes; therefore, there was limited statistical power to detect differences in these groups, and only large effects could be determined to be significant. Therefore, the findings in these specific subgroups should be interpreted with caution, mainly to provide hypotheses for future studies with larger samples. Finally, the relevant analysis revealed the correlation between audience engagement indicators and video quality. However, this correlation does not imply a causal relationship, and the observed association may be influenced by confounding factors. For instance, viewers might ‘like’ a video for reasons unrelated to its information quality, such as its entertainment value, production quality and the popularity of the video creators.
