3.2.1. Product attribute identification

We first filter out the most significant words from the preprocessed online reviews and maintenance records with the common TF-IDF algorithm (Salton and Buckley, Reference Salton and Buckley1988). Then, the Skip-Gram model (Guthrie et al., Reference Guthrie, Allison, Liu, Guthrie and Wilks2006) is leveraged to embed these words into a low-dimension vector space, and the obtained word vectors are clustered with the X-means clustering technique (Pelleg and Moore, Reference Pelleg and Moore2000). These clusters can be considered as the main product attributes identified. Based on the functional structure of the product (e.g., a motor vehicle usually consists of systems in power, chassis, body, and electronics), those attributes sharing the same product function will be merged (e.g., horsepower and acceleration both belong to the power attribute). In addition, those attributes with too low occurrence frequencies will also be merged into attributes with comparable occurrence frequencies and similar functional structures. Finally, the name of each cluster (i.e., identified product attribute) and the significant words in each cluster form an attribute-keyword dictionary. Note that the one significant word can only be found in one cluster (i.e., one product attribute). Thus, once a significant word from the attribute-keyword dictionary is detected from a text of either an online review or product maintenance record, the mentioned product attribute can be identified.

3.2.2. Semantic group generation

After obtaining the identified product attributes, the next step is to get customers’ attitudes toward these attributes. A tricky issue here is that customers occasionally express their opinions toward one product attribute in multiple sentences, and sometimes they mention multiple product attributes in one sentence. For example, power and fuel consumption may be mentioned simultaneously in a sentence from a customer’s review of vehicle performance. Thus, we cannot simply perform sentiment analysis on customer reviews by separating sentences with periods or commas, which will bring inevitable bias. To overcome this issue, we develop a semantic group generation method for customer opinion term extraction. Then, sentiment analysis can be applied to the generated semantic groups and extracted opinions to realize the aspect sentiment classification.

A semantic group is defined as a collection of descriptions of a single product attribute from one customer’s product review. Here, the descriptions can include one or multiple phrases and sentences. In other words, a semantic group is an attribute-description pair. One customer review can include one or more semantic groups. Then, when counting the identified product attributes and calculating sentiment scores, semantic groups can be considered as the most fundamental textual units, and customers’ attitudes and thoughts on products can be more accurately captured. The general process for semantic group generation is shown in Figure 2.

Figure 2. The general process for semantic group generation. The description separation approach in the orange box will be explained in Section 3.2.3.

Specifically, customer reviews are first split into short sentences consisting of words and phrases by segmentation, and part-of-speech (POS) tagging is applied to get the corresponding lexicality tags (e.g., nouns, verbs, etc.) of the words and phrases. Then, the nouns or noun phrases in each short sentence are searched in the attribute-keyword dictionary built in the previous step to find the corresponding product attributes mentioned by customers. Based on the number of attributes identified in each short sentence, semantic groups are generated in three ways.

1. 1. If no attribute is identified in the current short sentence, the last-mentioned attribute in the previous sentence of the review will be assigned as the attribute of the current sentence. This assigned attribute and the current sentence are combined as a semantic group. For example, in one review “This phone has a small battery capacity. It makes my user experience particularly poor,” there is no product attribute mentioned in the second sentence. Then, the second sentence will be added to the semantic group of the Power attribute, which is the last-mentioned attribute. If there is no last-mentioned attribute, the current sentence will not be considered.

2. 2. If there is only one attribute in the current short sentence, the semantic group is automatically generated by pairing the identified attribute and the current sentence.

3. 3. If multiple attributes are identified in one short sentence, dependency parsing is applied to separate the corresponding descriptions of each identified attribute. Then, each pair of the identified attributes and corresponding descriptions form a semantic group. The detailed procedures of this treatment are explained in Section 3.2.3.

After all semantic groups are generated from a customer’s review, those semantic groups describing the same product attribute will be merged.

3.2.3. Description separation for sentences with multiple identified attributes based on dependency parsing analysis

To accurately capture customers’ opinions and attitudes in their reviews where multiple attributes are identified in one sentence, we develop a dependency parsing-based method to separate the corresponding descriptions with respect to each identified attribute. Dependency parsing can provide the syntactic structure of sentences in terms of dependency relations. An open natural language processing platform, Language Technology Platform Cloud (LTP-Cloud) (Che et al., Reference Che, Feng, Qin and Liu2020, Reference Che, Li and Liu2010), is utilized in this operation, and 14 dependency relations can be identified. Their descriptions and tags are listed in Table 1. For example, ATT (Attribute) refers to the syntactic relationship between a noun and an attributive adjective or modifier. In the phrase “the vehicle’s engine,” “vehicle’s” is the attributive modifier that modifies the noun “engine.” Thus, an ATT relation can be identified between “vehicle’s” and “engine.” Verb-object (VOBs) refers to the syntactic relationship between a verb and its object in a sentence. For the sentence “the driver started the engine,” the VOB relation between “started” and “engine” can be identified. More detailed explanations of these relations can be found in (Che et al., Reference Che, Li and Liu2010).

Table 1. Dependency relations identified from dependency parsing analysis

Figure 3 shows an example of dependency parsing. The sentence is segmented as “[image]”, which means “This /phone /’s / battery /is very /durable”.Footnote ¹ Usually, subject–verb (SBV) relation is the most common relation between keywords (e.g., “battery”) and descriptions (e.g., “durable”), which are often adjectives. Also, the adverbial (ADV) relation modifies or qualifies the descriptions (e.g., “very durable”) and is quite useful in the sentiment analysis of sentences. Based on our preliminary study, the five most important dependency relations (SBV, VOB, ATT, ADV, and COO) are selected for further analysis.

Figure 3. An example of dependency parsing (the original sentence is in Chinese). One sentence can include multiple dependency relations.

Figure 4 illustrates the overall workflow of description separation for sentences with multiple identified attributes, which corresponds to the orange box in Figure 2. After dependency parsing, the obtained various dependency relations need to be analyzed one by one until all key product attributes with associated descriptors from the sentence are extracted, from which the semantic groups can be sequentially generated.

Figure 4. The overall workflow of description separation for a sentence with multiple identified attributes. The yellow blocks indicate the input and output of this process.

In this process, we first examine if two or more keywords of identified product attributes appear in an ATT relation, i.e., Case 1. In this case, the keyword at a lower functional level is retained as it usually provides more specific information. For instance, as shown in Figure 3, both “phone” and “battery” appear in the ATT relation, and we choose to keep “battery” as the keyword since it is more specific. After this treatment, if there is only one attribute in the sentence, a semantic group can be generated by pairing the attribute with this sentence. Otherwise, the sentence is passed to the examination of case 2.

In case 2, we check if there are multiple keywords of attributes in a COO relation of the sentence. For example, the keywords “fuel consumption” and “power” are in a COO relation in the sentence “The fuel consumption and power are surprisingly good for this vehicle.” We first count the number of adjectives in this relation. If only one or no adjective is mentioned in the relation, we treat this whole sentence as the description for all keywords. If the number of adjectives is two or above, for each keyword, the adjective closer to the first identified keyword is treated as its corresponding descriptor. Then, a semantic group can be generated by pairing the attribute with its corresponding descriptor. After that, this adjective will be deleted from the sentence to ensure the correct search and pairing of the remaining keywords and descriptors. If no COO relations are found, the sentence is then passed to the examination of case 3.

In case 3, there are no ATT or COO dependency relations between the keywords of attributes in a sentence. Thus, the descriptors can be obtained by searching the dependency relations for each keyword of the attribute separately. Before the search, keywords are first checked and updated to ensure they are complete. If the dependency relation between the keyword and the non-keyword is ATT, and the keyword is the modifier of the non-keyword, we consider updating the keyword by combining it with the non-keyword. For example, in the phrase “battery capacity,” “battery” is included in our keyword dictionary, while “capacity” is not. There is an ATT relation between “battery” and “capacity,” and “battery” is the modifier of “capacity.” Then, the keyword “battery” should be updated to “battery capacity” as the new keyword for the attribute “Power.” After the updating, the new keywords are used in the subsequent searches of relations such as SBV, VOB, and ADV to find the descriptors. The semantic groups are generated by combining the attribute with the corresponding descriptors.

In a word, the proposed workflow in Figure 4 aims to extract the corresponding descriptions on each mentioned attribute in a sentence of customer review as thoroughly as possible by checking various dependency relations of the keywords of attributes. The obtained descriptors are then paired with the product attributes to form semantic groups for further analysis.

3.2.4. Sentiment analysis

In this study, we use two sentiment analysis models to ensure the classification accuracy. The first one is ERNIE (Sun et al., Reference Sun, Wang, Li, Feng, Tian, Wu and Wang2020), which can learn semantic knowledge by modeling the words and entity relationships from massive data. The second one is the BiLSTM model, a combination of forward LSTM and backward LSTM (Xu et al., Reference Xu, Meng, Qiu, Yu and Wu2019). It is often used to model contextual information in natural language processing, which can better capture bi-directional semantic dependencies for accurate sentiment analysis. Both two models have been pre-trained using public datasets with high classification accuracy. ERNIE and BiLSTM can generate sentiment classification probability for each semantic group within a range of [0, 1]. To reduce potential biases and improve the model performance, we propose a voting mechanism to classify the final sentiment score of a semantic group into three categories (i.e., “positive,” “neutral,” or “negative”) based on the following rules:

1. 1. If $ {Pos}_1\ge 0.8 $ and $ {Pos}_2\ge 0.8 $ , the sentiment polarity of the semantic group is labeled as positive.

2. 2. If $ {Neg}_1\ge 0.8 $ and $ {Neg}_2\ge 0.8 $ , the sentiment polarity of the semantic group is labeled as positive.

3. 3. In other cases, the semantic group is not deterministically evaluated as positive or negative, and we mark it as neutral.

3.3.1. Estimation of performance

In this paper, the performance of a product attribute is estimated by two methods: calculating the sentiment scores of the attributes from online reviews or counting the occurrence frequencies of the attributes in product maintenance records. In the first method, for a dataset with $ R $ customer reviews, a review $ r $ ( $ r\in 1,2,\dots, R $ ) includes semantic groups for each identified product attribute $ i $ , where $ i $ is the index of the product attribute. After sentiment analysis, the semantic group will be assigned a sentiment polarity $ {S}_{ir} $ . We set “positive” as 1, “neutral” as 0, and “negative” as −1. Let $ {R}_i $ be the number of reviews that mention the attribute $ i $ . Then, the average sentiment intensity $ {S}_i $ of product attribute $ i $ can be calculated using Equation (1).

(1) $$ {S}_i=\frac{1}{R_i}\sum \limits_{r=1}^{R_i}\;{S}_{ir} $$

We propose to use $ \overline{S_i} $ as the metric of a product attribute’s performance as shown in Equation (2):

(2) $$ \overline{S_i}=\frac{S_i-\underset{i\in I}{\min}\;{S}_i}{\underset{i\in I}{\max}\;{S}_i-\underset{i\in I}{\min}\;{S}_i} $$

$ \overline{\;{S}_i} $ is a normalization result, and $ I $ is the set of indexes of all attributes. $ \underset{i\in I}{\max }{S}_i $ and $ \underset{i\in I}{\min }{S}_i $ represent the largest and smallest values of the $ {S}_i $ (i.e., the overall sentiment intensity of product attribute $ i $ ) among all attributes.

In the second method, the performance of a product attribute can also be measured by the occurrence frequency of its corresponding maintenance items (e.g., a fix job on a vehicle engine). More maintenance items associated with a product attribute imply a lower performance of this attribute from the customers’ real user experience. Equation (3) shows how to calculate the performance of a product attribute based on the occurrence frequency of its corresponding maintenance items. Equation (4) normalizes the result.

(3) $$ {P}_i=\frac{1}{M}\sum \limits_{m=1}^M{P}_{im} $$

(4) $$ \overline{P_i}=\frac{\underset{i\in I}{\max }{P}_i-{P}_i}{\underset{i\in I}{\max }{P}_i-\underset{i\in I}{\min }{P}_i} $$

Here $ {P}_i $ represents the average number of the maintenance items of attribute $ i $ in $ M $ maintenance records. $ {P}_{im} $ is the number of related maintenance items regarding product attribute $ i $ in maintenance record $ m $ . $ \overline{P_i} $ is the normalized result. Attributes with a smaller number of maintenance items will have lower $ {P}_i $ . After applying the conversion in Equation (4), an attribute with larger $ \overline{P_i} $ (i.e., closer to 1) has better performance.

3.3.2. Estimation of importance

In this study, the importance of a product attribute is estimated by the influence of the sentiments of this product attribute on customers’ overall product rating from online reviews. The extreme gradient boosting (XGBoost) model is leveraged to measure this influence due to its superior performance in modeling efficiency and prediction accuracy (Chen and Guestrin, Reference Chen and Guestrin2016). In the XGBoost model, a batch of trees is constructed with different features to estimate the final output. To construct the XGBoost model, customer sentiment scores for each attribute are used as the input, and the output is the overall rating of the product. To better fit this model, customers’ overall ratings on products are classified into two categories, “positive” or “negative.” After the training, the importance score for each feature is calculated using the gain-based feature importance estimation algorithm, and a 10-fold cross-validation is used to obtain better modeling results.

For a single leaf of the tree (i.e., a set of reviews grouped with the same classified label), gain measures the loss reduction after splitting this leaf into left and right leaves (i.e., splitting these reviews to two sets with different labels) using a feature (i.e., one of the product attributes). Define $ {D}_n=\left\{d|q\left({x}_d\right)=n\right\} $ as the indices of data assigned to leaf $ n $ . Gain for leaf $ n $ to be split using attribute $ i $ is calculated as

(5) $$ {\displaystyle \begin{array}{c} Gain=\frac{1}{2}\left[\frac{{\left({\sum}_{d\in {D}_L}{u}_d\right)}^2}{\sum_{d\in {D}_L}{h}_d+\lambda }+\frac{{\left({\sum}_{d\in {D}_R}{u}_d\right)}^2}{\sum_{d\in {D}_R}{h}_d+\lambda }-\frac{{\left({\sum}_{d\in D}{u}_d\right)}^2}{\sum_{d\in D}{h}_d+\lambda}\right]-\gamma \\ {}{u}_d={\partial}_{{\hat{y}}_d^{\left(t-1\right)}}l\left({y}_d,{\hat{y}}_d^{\left(t-1\right)}\right)\hskip1em and\hskip1em {h}_d={\partial}_{{\hat{y}}_d^{\left(t-1\right)}}^2l\left({y}_d,{\hat{y}}_d^{\left(t-1\right)}\right)\end{array}} $$

where $ {u}_d $ and $ {h}_d $ are first computed to simplify the calculation. $ l $ is the loss function, and $ t $ stands for the iteration steps. $ {D}_L $ and $ {D}_R $ are the data sets in the left and right leaves. $ \lambda $ and $ \gamma $ are the controlling factors to avoid overfitting (Chen and Guestrin, Reference Chen and Guestrin2016). The feature importance $ {Imp}_i $ for attribute $ i $ is then defined as the average gain over all trees and leaves with attribute $ i $ as the splitting feature (Loh, Reference Loh2011). Let $ K $ be the number of constructed trees. For a single tree $ k $ , there are $ {N}_k-1 $ internal nodes. Then, the importance is calculated as

(6) $$ {Imp}_i=\frac{1}{K}\sum \limits_{k=1}^K\sum \limits_{n=1}^{N_k-1}{Gain}_{kni} $$

where $ n $ is the index of the node. The attribute $ i $ will have higher feature importance if it contributes more to the prediction performance of the model (Zheng et al., Reference Zheng, Yuan and Chen2017). The final importance $ \overline{Imp_i} $ of attribute $ i $ is calculated with the normalized value of $ {Imp}_i $ as shown in Equation (7).

(7) $$ \overline{Imp_i}=\frac{Imp_i-\underset{i\in I}{\max}\;{Imp}_i}{\underset{i\in I}{\max}\;{Imp}_i-\underset{i\in I}{\min}\;{Imp}_i}. $$

Attribute occurrence frequencies from online reviews can also be calculated to reflect their importance. These results are then compared with the importance results obtained by the XGBoost classification model to present a more comprehensive understanding of attribute importance.

3.3.3. Importance-performance plot

After getting the importance and performance of each product attribute, the IPA plot can be generated by marking the performance and importance of product attributes in a plane rectangular coordinate system. Then, the mean values of the performance and importance over all attributes are calculated to divide the plot into four quadrants, which are labeled as “Keep up the good work,” “Concentrate here,” “Low priority,” and “Possible overkill” from quadrant I to quadrant IV in a counterclockwise manner. Designers can then take tailored actions for the product attributes falling in different quadrants. For example, attributes in quadrant II (“Concentrate here”) have low performance but high importance, which means a high priority for design improvement. However, evaluating the improvement priorities only by quadrants may bring difficulties for designers to evaluate those attributes quite close to the intersection boundary of different quadrants (Azzopardi and Nash, Reference Azzopardi and Nash2013). For those attributes close to the quadrant II in an IPA plot or attributes with extremely low performance (i.e., the attribute with the lowest performance value among all attributes), we also highly recommend designers pay more attention to them. In this research, an attribute in other quadrants is suggested to be close to quadrant II if it has a performance value smaller than 1.05 times the average performance and an importance value larger than 0.95 times the average importance. These two coefficients are obtained from our pilot tests as they allow most important attributes around quadrant II to be covered. In practice, designers can adopt more lenient criteria if they intend to make more aggressive design improvement strategies.

3.4.1. Validation by comparing IPA results with actual product improvements on the market

The first validation method is to check whether the actual product improvements on the market agree with the suggestions from the IPA results. Since a major product attribute usually contains various components (e.g., the multimedia system of a vehicle includes screen, audio, and other subsystems), the upgrading status of each major product attribute should not be simply determined as 0 or 1 (i.e., not improved or improved). A more precise improvement score is needed based on the improvement of the components under this attribute.

We first select related semantic groups with negative customer sentiment scores for each major product attribute $ j $ . Then, in these semantic groups, attributes’ related keywords (i.e., components) are extracted, and their mentioned frequencies are calculated. The same keywords mentioned in one semantic group are only counted once. The $ W $ most frequently mentioned keywords for each attribute are chosen for further analysis. Let $ {F}_{wj} $ ( $ w=1,2,\dots W $ ) represents the mentioned frequency of the keyword $ w $ of attribute $ j $ . Then, the weight of keyword $ w $ is calculated as shown in Equation (8):

(8) $$ {a}_{wj}=\frac{F_{wj}}{\sum_{w=1}^W{F}_{wj}} $$

$ {a}_{wj} $ represents the contribution of keyword $ w $ to customers’ negative reviews on attribute $ j $ . Keywords with higher occurrence frequencies in negative semantic groups have larger weights, which indicates that these keywords/components are likely to be the more urgent design improvements from customers’ point of view. After that, by examining the real improvements of the product on the market, the flag of component improvement $ {g}_{wj} $ can be determined, in which $ {g}_{wj}=1 $ represents the component $ w $ of attribute $ j $ has been improved on the market, otherwise, $ {g}_{wj}=0 $ . Therefore, the total improvement score of attribute $ j $ is calculated as shown in Equation (9).

(9) $$ {p}_j=\sum \limits_{w=1}^W\;{a}_{wj}{g}_{wj} $$

Here $ {p}_j=1 $ means all of the main components of the product attribute $ j $ have been improved in the real market, while $ {p}_j=0 $ means none of its components are improved.

3.4.2. Validation by comparing the analysis results of online reviews and maintenance records before and after product improvements

In the second validation method, IPA results based on online reviews and maintenance records for products in different model years are analyzed and compared.Footnote ² We first perform IPA using customers’ online reviews for the same product in two model years (i.e., before and after improvements). Then, we calculate $ \Delta {S}_i $ , the change of the average sentiment scores for attribute $ i $ as shown in Equation (10):

(10) $$ \Delta {S}_i={S}_i^{\prime }-{S}_i $$

Here $ {S}_i $ and $ {S}_i^{\prime } $ represent the average sentiment scores for attribute $ i $ before and after the improvements with a range of $ \left[-1,1\right] $ . $ \Delta {S}_i $ can be used to measure the change in customers’ satisfaction, and larger $ \Delta {S}_i $ shows greater improvement in attribute $ i $ from customers’ experience.

To measure the change in customers’ satisfaction with product attributes in a more accurate way, we adopt the same calculation method as described in Section 3.4.1. First, the frequency $ {F}_{wj} $ for the keyword $ w $ of the attribute $ j $ in negative semantic groups is counted. Then, the proportion of occurrence frequency of keyword $ w $ before the improvement can be calculated as shown in Equation (11):

(11) $$ {c}_{wj}=\frac{F_{wj}}{\sum_{j=1}^J{\sum}_{w=1}^W{F}_{wj}} $$

$ J $ is the total number of product attributes. Therefore, $ {c}_{wj} $ represents the contribution of keyword $ w $ to all negative reviews about the product from customers. Similarly, the occurrence proportion $ {c}_{wj}^{\prime } $ for keyword $ w $ in the negative semantic groups from the online reviews of the improved product can also be evaluated. $ {c}_{wj} $ and $ {c}_{wj}^{\prime } $ range from 0 to 1. Then, the improvement scores of keywords and the attribute are evaluated using the percentage of change for occurrence proportions in two model years as shown in Equations (12) and (13).

(12) $$ {p}_{wj}=\frac{c_{wj}-{c}_{wj}^{\prime }}{c_{wj}}\times 100\% $$

(13) $$ {p}_j=\sum \limits_{w=1}^W\;{a}_{wj}{p}_{wj} $$

Here $ {p}_{wj} $ is the improvement score of the keyword $ w $ within product attribute $ j $ , and $ {a}_{wj} $ is the weight of the keyword $ w $ calculated by Equation (8). Also, $ W $ most frequently mentioned keywords for each attribute before improvements are selected. Then, $ {p}_j $ , the improvement score for attribute $ j $ is calculated by using the weighted sum of the improvement scores for these keywords. Positive improvement scores indicate improvements in customers’ satisfaction, while negatives mean worse experiences. The closer the improvement score to 1, the fewer complaints from customers about the product attributes. The validation using product maintenance records is similar to the above procedure, and the only difference is that the attribute frequency $ {c}_{wj} $ and $ {c}_{wj}^{\prime } $ is counted within all maintenance records when using Equation (11).

6.2.1. Validation using online reviews before and after product improvements

In this validation method, customers’ online reviews on SUV A of 2018 and 2022 are analyzed and compared. Following the procedures introduced in Section 3.4.2., the vehicle attributes are sorted by the change in performance $ \Delta {S}_i $ (refer to Equation (10)) from 2018 to 2022. Table 8 shows the sorted results and the change in customer satisfaction, and the rankings of these changes are labeled in the importance-performance plot of Figure 9. We find that for most of the vehicle attributes, customers tend to have a higher evaluation of the model in 2022. Attributes with low performance but high importance have been improved most (e.g., electronics and multimedia are in quadrant II, and their customer sentiment score changes are 0.358 and 0.263 with a rank 2 and 3, respectively). The improvements of attributes with low performance are usually larger than those with high performance (e.g., most top-ranking attributes are in quadrants II and III). The three most improved attributes are light, electronics, and multimedia, which is consistent with the suggestion from our IPA results in Section 5.2. Light performs worst with a normalized performance of 0, but has the largest change with the value of 0.719. The fourth most improved attribute is intelligence, which conforms to the rapid development of smart technologies for automobiles in recent years (Hu et al., Reference Hu, Lou, Xing, Wang, Cao and Lv2022).

Table 8. Changes in customer sentiment scores for SUV A from 2018 to 2022 based on online reviews. $ \Delta {S}_i $ is the customers’ average sentiment change of attribute $ i $ (see Equation (10))

Figure 9. The importance-performance plot for SUV A in 2018 obtained from online reviews with rankings of changes in customer sentiment scores for these attributes.

Then, the improvement scores for three vehicle attributes with urgent need of improvements and their five most frequently mentioned keywords are reported in Table 9. Similar to the results obtained in Sec. 5.1, Light still has the largest improvement score close to 1. Both electronics and multimedia receive good scores for improvements, which implies customers make fewer complaints on these attributes in 2022.

Table 9. Improvement scores for three vehicle attributes obtained from online reviews within two model years. $ {p}_{wj} $ is the improvement score of keyword $ w $ from attribute $ j $ . $ {p}_j $ is the total improvement score of attribute $ j $

Figure 11 presents the importance-performance plot obtained from online reviews in 2022, and the IPA plot using online reviews in 2018 is overplayed in Figure 10 for easier comparison. We find that attributes that mostly need to be improved change to interior, intelligence, comfort and maintenance. The performance of light and multimedia increases above the average. The performance of the electronics also increases but its importance becomes very low. The importance of intelligence increases a lot, which implies that customers are paying more attention to the Intelligence functions such as parking assistance and car networking. These results indicate that the proposed IPA framework is applicable to actual product improvement planning.

Figure 10. The importance-performance plot for SUV A in 2018 obtained from online reviews.

Figure 11. The importance-performance plot for SUV A in 2022 obtained from online reviews.

6.2.2. Validation using maintenance records before and after product improvements

According to the IPA results using maintenance records in Section 5.3, attributes that mostly need to be improved are Electronics, Structure, and Control. Table 10 shows the changes in occurrence frequencies for these attributes. We find that the proportion changes for most attributes are relatively small. In Figure 13, the IPA plot obtained from maintenance records with rankings of occurrence frequency proportion changes is presented. Compared to the labeled plot obtained from online reviews in Figure 12, the results from maintenance records are more irregular. Except for Electronics, Structure, and Operation, the changes in occurrence frequencies of other attributes are more likely to fluctuate. One possible explanation is that most of the maintenance records for improved products come from 2019 due to the limited data source, then the attribute performance is very close to the original product (i.e., SUV A in 2018). Thus, the improvement scores may not be as obvious as those obtained from online reviews.

Table 10. Occurrence frequency proportion change for vehicle attributes obtained from maintenance records from 2018 to 2019. $ \Delta {P}_i $ is the occurrence frequency change for attribute $ i $ (refer to equations (10) and (13))

Figure 12. The importance-performance plot for SUV A in 2018 obtained from online reviews with rankings of changes of customer sentiment scores for these attributes.

Figure 13. The importance-performance plot obtained from maintenance records in 2018 with rankings of occurrence frequency proportion changes of attributes.

Figure 14. The combined importance-performance plots with $ \alpha =0.5 $ (upper) and $ \alpha =0.7 $ (lower).