Home on Xijia Che

Confirming Claims of Superposition and Adversarial Examples in Toy Models

Mon, 27 Jul 2026 00:00:00 +0800

Superposition: when a model has fewer dimensions than the data has features, it cannot give each feature its own direction — projecting them into the bottleneck forces them to share axes and interfere.

tl;dr #

We reproduce all three core claims of Stevinson et al. from their toy classifier setting:
- PGD attacks against toy models generally agree with theoretically optimal solutions.
- Toy models without superposition are less vulnerable to attacks. Robustness falls monotonically as superposition increases.
- Attacks transfer between independently-trained models because feature geometries can converge across training runs.
Similar to what’s reported by Elhage et al., 2022, we notice that co-activating features can collapse onto a shared direction instead of orthogonalizing.

Our code can be found here and we encourage readers to fork/run followup experiments!¹

Background #

Elhage et al., 2022 defines superposition as the phenomenon where “models represent more features than they have dimensions.” To study this in a toy setting, they define a classifier where $k$ features are squeezed into an $m$-dimensional bottleneck with $m < k$, so feature directions interfere in the latent space.

Stevinson et al., the origin of the experiments in this replication, argues this interference is a mechanism behind adversarial examples. The paper makes three claims: 1. adversarial attacks exploit interference, 2. models without superposition are more robust to adversarial examples, 3. correlations in the input data allow models to converge to similar geometry, allowing adversarial attacks to transfer between models.

Toy Model and Notation #

The experiments in the paper deal with a toy model with encoder $W_e \in \mathbb{R}^{m \times k}$ and decoder $W_d \in \mathbb{R}^{k \times m}$. For input $x \in \mathbb{R}^{k}$ the output is $y = W_d W_e x \in \mathbb{R}^{k}$. Here $k$ is the number of classes and $m < k$ is the bottleneck width forcing the features to be condensed into a lower dimensional space before reconstruction. This gives us a convenient architecture to study superposition: a large ratio $k/m$ means the model has to do more compression, while if $k/m = 1$, the model can represent each feature with its own direction.

The model uses cross entropy loss. Features are sampled independently with probability $1 - s$, where $s$ can be thought of as how sparsely features are activated within the data.

Adversarial attacks are computed by projected gradient descent (PGD) (Madry et al., 2019). For a given sample $x$, the adversarial example $x_\text{adv}$ is found by PGD, where $\delta = x_\text{adv} - x$ indicates how the attack modified the original sample. $\|\delta\|_p$ measures distance between the original example and the adversarial example and $\varepsilon$ is the attack budget which enforces that $\|\delta\|_p \leq \varepsilon$. We use PGD learning rate $\varepsilon/4$, at 50 steps with 5 restarts, and $\varepsilon = 0.2$. The values for $k$ and $m$ vary but unless otherwise stated $k = 7$ and $m = 2$.

We follow the hyperparameters described in the paper as of July, 2026.²

Attacks Exploit Interference #

Stevinson et al. hypothesize that adversarial attacks in toy models work by exploiting the interference geometry. Proposition 1 from the paper shows the optimal perturbation has a closed form solution:

$$\delta \propto W_e^\top (w_t - w_s) \tag{1}$$

where $w_s, w_t$ are the source/target class vectors (rows of the decoder $W_d$) and $W_e$ the encoder, scaled to the $\varepsilon$-ball. The authors test if the PGD perturbation matches the optimal attack derived from interference geometry, empirically by:

Performing untargeted attacks on the toy model.
Taking the set of successful attacks (an attack is successful if the prediction flips and the true label stays unchanged).
Finding the optimal targeted attack for target class $y = W_d W_e x_\text{adv}$ using Eq. (1) above and calculating the cosine similarity with $x_\text{adv} - x$.

At a high level, this can be thought of as letting the attack maximize loss in an arbitrary direction and testing if it finds an optimal direction for a given target class. Figure 1 shows the untargeted attack live where $k = 7$ and $m = 2$. Each arrow is one PGD step in the encoder’s latent plane, the dashed outline is the reachable set (the encoder image of the $\varepsilon$-ball), and the shaded regions show which class the decoder assigns (the attack succeeds when the activation is pushed across a decision boundary).

Figure 1: A projected-gradient-descent attack on the 2-D bottleneck of a toy superposition model, trained and attacked live in your browser.

We use untargeted attacks for these experiments to follow the methodology of the paper, but we also repeat this experiment for targeted attacks with results in Appendix A.

The claim that the PGD attack aligns with the optimal attack, exploiting the interference geometry, replicates cleanly on the toy model. With sparsity 0.98, we demonstrate that PGD attacks align with optimal perturbation. We believe our cosine similarity results are higher since we let PGD iterate to find the best perturbation while the paper stops at the first prediction flip.

k	m	clean acc	PGD·optimal (ours)	PGD·optimal (paper)
6	2	0.998	1.00	0.97
30	10	0.996	1.00	0.96
90	30	0.966	1.00	0.92

Like the original paper, we ablate on randomness by calculating the cosine similarity between our optimally successful PGD attack and a random direction within the $\varepsilon$-ball, and confirm on average that the cosine similarity between random directions and our PGD attack lands around 0.

As an additional stress test, we repeat our ablation while only selecting random directions that result in a successful attack, isolating the comparison between the PGD attack and random, successful attacks. These can yield non-zero cosine similarity scores, but the PGD attacks still align better with the optimal solutions.

k	m	random direction	random direction filtered by successful attack
6	2	0.015±0.028	0.639±0.006
30	10	0.003±0.007	0.432±0.003
90	30	0.002±0.002	0.213±0.07

No Superposition, Little Vulnerability #

Stevinson et al. claim that toy model adversarial vulnerabilities disappear along with superposition. We observe the same trend with robust accuracy falling monotonically as superposition pressure rises. When $k = m$ (all features can be orthogonal), robust accuracy grows to 97.3%.

Some of the magnitudes we report diverge. At $m = 2$ our model stays more robust than the paper’s because we train for 20k steps while the paper only trains for 5k steps. At $m = 5$ our robust accuracy is lower than the paper since we give PGD random starting positions, bigger steps, and 5 restarts to find a better attack, which is a setting stated in the paper but missing in the authors’ code.

Robust accuracy vs k/m
k/m	paper m=2	released code m=2	ours m=2	paper m=5	released code m=5	ours m=5
1	98.7	98.6	100	92.0	95.0	97.3
2	75.8	80.1	79.5	87.2	97.0	70.5
3	31.0	34.3	50.4	59.0	66.4	50.6
4	11.1	11.4	18.7	33.5	36.7	30.1

Shared Geometry Drives Transfer #

Stevinson et al. claim that input correlations shape the learned feature geometry, and attacks transfer between independently-trained models to the degree their geometries converge.

We train independent models under three correlation regimes:

Uncorrelated: features activate independently
Paired: paired classes activate at the same time
Global: adjacent classes frequently co-activate, specifically:
- Each sample draws a phase $\varphi$; class $c$ activates with probability $0.5 + 0.5\cos(\varphi + 2\pi c/k)$.
- Active classes get $\text{Uniform}(0, 1)$ values; the prediction label is given by argmax.
- This process makes index-neighbors more likely to activate together. This incentivizes geometry to converge across seeds.

Figure 2: Sampling in the global regime. Each sample draws a phase which sets a cosine bump of activation probabilities over the classes (left); active classes draw uniform values and the label is the argmax (middle); across samples the bump rotates with the phase, so index-neighbors co-activate (right).

We see the same trend where both geometric similarity and attack transfer rate grow as the correlation of input increases.

Figure 3: Geometric similarity and targeted attack transfer rate per regime, ours vs paper.

Figure 4 shows the geometry between the two seeds. As expected, uncorrelated arrangements are random to each other and unrelated between models.

For the other two settings, we observe an interesting phenomenon: correlated features tilt toward each other rather than orthogonalizing. We dive into it in the next section.

Figure 4: Class-vector geometry for two seeds across the three regimes. In (b) co-activating partners share a color and a collapsed pair is haloed; in (c) the dashed loop links the cyclic index-neighbors.

Similar geometry and attack transfer are verified on regimes with larger input space. As we stated above, there is a magnitude gap for the paired and global settings due to different parameter choices.

Beyond Replication: Orthogonalize or Collapse? #

What do co-activating features do to each other’s geometry? We sweep the paired regime’s coupling probability $t$ (partner fires with probability $t$ when its mate fires) across bottleneck widths.

When $m = 2$, as the coupling probability $t$ rises, partners swing together and collapse onto a shared axis, until every pair has merged into roughly three directions. With capacity slack ($m = 3, 4$) partners stay near-orthogonal or in avoidance at every coupling; at $m = 2$, partner cosine crosses above the baseline near $t \approx 0.75$ which is the collapse regime.

This tells us latent space geometry is constrained by paired activation. With sufficient model capacity, paired features tend to orthogonalize; however, when capacity is lacking, they collapse to a fixed angle.

Figure 5: Partner cosine (blue) vs the non-partner baseline (grey) as coupling t rises, across 5 seeds. m = 3, 4 stay near-orthogonal; at m = 2 the partner cosine crosses above the baseline near t ≈ 0.75 — the collapse regime.

Appendix A #

We perform targeted attacks on every ordered class pair $s \to t$ (where we minimize the cross entropy loss of the target class). We verify that these attacks align with the optimal direction with cosine similarity higher than 0.96.

This holds for both successful and failed attacks. Even when $m = k$, where orthogonal geometry prevents vulnerability, the attacks precisely match the optimal perturbation but fail to breach the decision boundary.

We demonstrate that attack success tracks head interference as shown in Figure 6. Geometry predicts vulnerability pair by pair, not just on average.

Figure 6: When features are superposed, attack success rate increases as head interference grows stronger.

This research is supported by the Second Look Research fellowship. ↩︎
For training details that weren’t specified, we make reasonable assumptions and in cases where results looked qualitatively different, we investigated further or contacted the authors. ↩︎

The Boundless Voice of Night

Mon, 02 Feb 2026 00:00:00 +0800

This is a translation by Claude. The original Chinese version is below.

“The wind wanders everywhere, stringing together the night’s messages, from one sleeping window to another, calling on the moods that daylight overlooked. Another kind of world, lush and thriving—the night’s voice is boundless.”

Night is when ghosts and monsters come out to wander. The rationality, the deliberation, the fences we build up through the day all come collapsing down once deep night falls. That’s why I’m afraid of the dark—I always have to be asleep before eleven.

Today I decided to listen to the night. To listen to the Temple of Earth in Shi Tiesheng’s writing—four hundred years in which it stripped away the ostentatious glazed tiles from the ancient hall’s eaves, faded the vermilion that once flaunted itself on the gates and walls, let long stretches of high wall crumble and jade balustrades scatter, while the old cypresses around the altar grew ever more somber and green, and the wild grass and vines everywhere grew lush and unrestrained, at ease with themselves.

Today I decided to listen to the night. To listen to a water pipe bursting, to heavy snow spilling down and coating the streets in thick ice. To listen to an infant crying as it nurses on the other side of the earth, to a rooster’s crow rising with the first cooking smoke, to a barefoot boy, backpack slung over his shoulders. To listen to you, to her, to the rain, to the night.

“风，四处游走，串联起夜的消息，从沉睡的窗口到沉睡的窗口，去探望被白昼忽略了的心情。另一种世界，蓬蓬勃勃，夜的声音无比辽阔。”

夜晚是鬼怪出没的当口，白日里构建的理智、思考和围栏，在深夜里统统溃散。所以我惧怕黑夜，十一点前总要进入梦乡。

今天决定听夜。听史铁生笔下的地坛，四百多年里，一面剥夺了古殿檐头浮夸的琉璃，淡褪了门壁上炫耀的朱红，坍圮了一段段高墙又散落了玉砌雕栏，祭坛四周的老柏树日渐苍幽，到处的野草荒藤也都茂盛的自在坦荡。

今天决定听夜。听水管撕裂，大雪漫下，在街边涂上厚厚的冰。听地球另一边啼哭着吃奶的婴孩，听炊烟刚起时伴着的鸡鸣，听背着书包的赤脚少年，听你，听她，听雨，听夜。

My First Semester of a PhD in America

Mon, 22 Dec 2025 00:00:00 +0800

This is a translation by Claude. The original Chinese version is below.

Some say a PhD is an endless summer of life. The loneliness of walking alone at the cognitive frontier of human civilization lets us feel the dignity and nobility of being “Homo sapiens.”

In my first semester of the PhD, I hadn’t yet felt that nobility, nor could I keep my dignity while racing to catch up on assignments. Maybe my solitary pace can no longer keep up with the development of computer science. Maybe I’m not Homo sapiens after all.

August. I left home in the Northeast, and for the last time before leaving the country, boarded a high-speed train, passing through Shanhaiguan to reach Beijing. Beijing is something like my second hometown—after seven years living there, I’m still stunned by its vastness. Beijing’s bigness isn’t just that everywhere you go from Haidian takes an hour on the subway—it’s even more so the city’s stillness, its depth, its indifference. That night I drove from Chaoyang Station up onto the elevated highway, half-lit buildings receding on either side, nothing in my ears but the rush of wind.

Beijing hasn’t changed at all since I first moved there seven years ago.

I felt small. In this city, the reckless shouts of my youth left no echo, and the ink I flung out with all my strength barely left a mark. I spent most of my youthful energy here, but Beijing was like a girl I chased shamelessly for years—when I left, she didn’t try to keep me for even a moment.

The flight went through Dubai, then landed in Chicago, and by the time I reached Champaign it was already late Sunday night. I couldn’t get my room key, so I slept on the apartment couch. Because of the time difference, I woke up again just a few hours later, the sky barely starting to gray.

September, my first month of the PhD. I sized up the school with curiosity. UIUC is jokingly called “the cornfield”—the campus straddles the two small towns of Champaign and Urbana, and a ten-minute drive out from the center of campus drops you straight into the endless cornfields of the American Midwest.

There are no museums here, no shopping malls or skyscrapers, none of the dazzling shelves of goods that neoliberals love to talk about. It feels more like a footnote to the age of globalization, planted in the middle of America—the campus buses carry signs in English, French, and Chinese, there are quite authentic Asian restaurants and Chinese barbershops, and even Zhang Liang and Yang Guofu malatang.

When our lab drove to Indiana for a conference, the cornstalks along the highway looked like a few drops of paint melted out of the sunset, staining the horizon a grayish yellow. I opened my eyes wide and marveled, “So there really is nothing out here.” A senior labmate added, “No, no—actually there are a lot of animal carcasses. Cows, deer, rabbits, they’re pretty common around here.” I felt a kind of disorientation, like the scene of Lei Jiayin arriving in Beijing and seeing an ostrich running down the street in the middle of the night. All I could do was sigh that this continent had its own peculiar flavor.

Newly arrived in America, I had to plug myself into this unfamiliar society—setting up a phone number, a social security account, a bank account, an employment contract, health insurance paperwork, and so on. Cards and letters kept arriving one after another; one of them had a Mickey Mouse design on it, and I loved it. These newly arriving objects kept reminding me that my story was slowly unfolding here.

October. International students often complain about how poor the service staff’s attitude can be. I ran into one myself.

I can’t bring myself to excuse this DMV employee with the thought that “maybe he’s got his own troubles.” When my number was called to register my driver’s license information, I walked up to the counter and waited, but he kept right on chatting with a coworker beside him as if I weren’t there. Then, unhurried, he turned his head, his gaze sweeping over the rim of his glasses up and down a female coworker walking past, licking his lips, making no effort at all to hide—if anything, flaunting—his lechery. As he registered my information his tone dripped with sarcasm; even as a non-native speaker, I could feel the mockery and contempt in his voice.

It struck me that the meaning of his job, to him, wasn’t serving residents at all—it was getting to flirt with a female coworker, lord it over the people who needed something from him, wring out a little pleasure, satisfy some appetite, and burn through another day.

Apart from that, the people I’ve met here have mostly been practical or warm. Whether in America or China, East or West, all ordinary people can really do is tend to their own lives.

November. I used the fall break to visit Chicago. I didn’t plan much of an itinerary—when I ran into parents with kids at a museum café, I’d sit a while longer, munching chips and listening to the little humans talk about dinosaurs and stick insects. I sat by Lake Michigan until the sky began to dim, then got up and walked across half the city, snapping photos of whatever caught my eye along the way.

I walked through a long tunnel, and only after going deep inside did I realize there were no pedestrians in it, only cars passing back and forth. In front of every vent blowing out warm air lay a homeless person; I couldn’t tell whether what covered them was a blue blanket or plastic sheeting from a construction site. Some of them smiled at me, and I smiled back.

What stuck with me most was a middle-aged man I met at a station. I’d overshot my stop on the bus to the Field Museum, and it turned out the bus went barreling down the highway for another eight minutes before it reached a transfer station. Another station worker there, hearing where I was trying to go, looked at me and laughed, “Hey, this guy’s trying to get to the Field!”

The reason I can only describe him as “a middle-aged man I met at a station” is that he was, like me, about as ordinary as a person gets—wearing a red work jacket, a backpack on his shoulders, no hat, no gloves.

He kept pointing at the stop not far off, telling me over and over which bus to take. “Make sure you pull the cord after the bus gets on the highway. Get off, walk east, cut through the park, and you’ll be at the Field.” After I got on, he stood by the front door and told the driver, “He’s headed to the Field—remember to call his stop.” I noticed his teeth weren’t quite straight, which in Western culture is a clear class marker.

As someone who can put away another bowl of rice just from smelling food, even when I’m not hungry, one of the main points of any trip for me is filling my stomach.

I ordered Chicago deep-dish pizza because of its reputation, but sadly, the classic recipe is just tomato and cheese—after two slices the sourness and grease were too much, and I had to box up the rest. Stepping outside, snow suddenly began drifting down between the buildings. I told my Lyft driver how lucky I felt to catch snow on my first trip to Chicago, but he couldn’t quite understand my excitement, and asked if I was from somewhere like Florida where it never snows.

I wanted so badly to tell him that my hometown gets plenty of heavy snow too, and that as a kid I used to lie down in snowbanks and slide down them again and again.

December. My first PhD semester came to an end. It hit me all at once that I hadn’t produced any results—the fairly simple project assigned at the start of the semester had barely moved forward. My advisor is cheerful, considerate, and respectful of her students; she rarely pushed me, and patiently explained her thinking to me instead.

It turned out my whole research methodology had problems, and I’d spent a whole semester muddling through without noticing. On some nights, anxiety crept over my body; walking home from the lab, I wanted to gulp down a little more cold air, but was afraid of wandering into an unsafe block, so with nowhere to go, I’d just stand by the roadside pretending to wait for a ride—a genuinely ridiculous sight.

I was constantly on edge. My schedule wasn’t as packed as it was in college, but mentally I was unusually busy—almost cluttered, really.

For a while, I stuffed every scrap of spare time with podcasts. I swapped my morning alarm for Apple News; the half hour of breakfast and walking to my desk, the hour of lunch and the after-lunch walk, the time at the gym in the evening—podcasts played through my earbuds the whole way. Healing content, comedy, philosophy, AI, politics—I listened to whatever, with no rhyme or reason.

I also bought some paperback books—they’re not common back home; the only ones I’d read were copies my high school English teacher, Laura, bought for the class. Back then we read Animal Farm and A Thousand Splendid Suns, and I mostly just flipped through them to get Laura’s assignments done. Looking back now, that feels like a real regret.

At school, I was lucky enough to find two Chinese-language book clubs, which have brought a little joy to life in the cornfield. One roams freely through essays and poetry—from Camus and The Matrix to a Street Fighter game adapted from a sci-fi novel, to food blogs dug up from some corner of the internet. The other engages actively with social reality—listening to Hsu Cho-yun talk about America, and discussing how the arrogance of meritocracy swept away the glittering promises of some golden age.

These lovable, earnest friends keep reminding me that the far-off world is always full of allure, but here in this quiet cornfield, even if we can’t throw our whole bodies into the turbulent times unfolding around us, we can still reach out and explore.

有人说，读博是人生的无尽夏。在人类文明的认知边缘踽踽独行的孤独感，让我们体会到作为“智人”的尊严和高贵。

读博的第一学期，我尚未感受到这份高贵，也未能在追赶作业时保持尊严。或许我独行的速度已追不上计算机科学的发展，或许我并非智人。

八月，我离开东北的家，出国前最后一次坐上高铁，穿越山海关来到北京。北京算是我的第二故乡，在这里生活七年后，我依然为它的广大所震撼。北京的大，不只是从海淀去哪里都要坐一个小时的地铁，更是它的沉静、深厚和冷漠。那晚，我开车从朝阳站出发走上高架，两旁半明半暗的建筑不断在视野里后退，耳边只剩呼呼的风声。

北京与七年前我第一次搬来时相比，没有任何变化。

我觉得自己渺小，在这座城市里，青春时肆无忌惮的呐喊没有回声，用力撒出的墨也没沾染下几点痕迹。我少年的气力消耗了大半，可北京就像是我死皮赖脸追的姑娘，我离开时，她没有一丝挽留。

飞机辗转到迪拜，再落地芝加哥，等到香槟镇时已经是周日的深夜。我没拿到房间钥匙，就在公寓沙发上睡了一夜。因为时差，几小时后我醒来时，天才蒙蒙亮。

九月，博士的第一个月，我揣着好奇打量学校。伊利诺伊香槟分校（UIUC）被人戏称为“玉米地”，学校横跨在香槟和厄巴纳两座小镇中间，从校园中心朝外开十分钟车，就会走进美国中部一望无际的玉米田中。

这里没有博物馆，没有购物中心和摩天大楼，没有新自由主义者口中琳琅满目的商品货架。它更像是全球化时代在美洲中部的一处注脚，校园巴士上印有英语、法语、中文三种提示，有相当地道的亚洲餐馆和华人理发店，还有张亮和杨国福麻辣烫。

课题组去印第安纳开会时，高速路旁的玉米杆子，像是从夕阳融化出的几滴颜料，把地平线染成了灰黄色。我睁大眼睛感慨：“原来这里真的什么都没有。”高年级师兄补充说，“不不，其实有很多动物尸体，牛啊、鹿啊、野兔啊，在这里很常见。”我当时有种“雷佳音刚到北京看到鸵鸟大半夜在街上跑”的错位感，只能感叹美洲大陆别有一番滋味。

初来美国，我需要把自己接入这个陌生的社会，建立电话号码、社保账号、银行账户、工作合同、医保协议等等档案。我陆续收到一些卡片和信件，其中一张印有米老鼠图案，我非常喜欢。这些新到来的物件不断提醒我，我的故事已慢慢在这里展开。

十月，留学生时常感叹服务人员态度很差，我也遇到了一个。

我无法从“他或许也有难处”的角度，为这位车管所工作人员开脱。登记驾照信息时，系统叫到了我的编号，我走到柜台前面等候，他却依然视若无睹地和旁边的同事聊天。再不慌不忙地扭过头，目光透过眼镜上沿，上下扫荡着走过的女同事，舔舔嘴唇，毫不掩饰甚至是袒露自己的淫荡。登记信息时阴阳怪气，我不是母语者，也能感受他语调里的嘲弄和不屑。

我感到，他工作的意义不是为居民提供服务，而是能与女同事搭讪，对有求于他的人施以欺压，赚取快感，满足欲望，然后消耗掉一天。

除此以外，我遇到的人们往往务实或热情，无论在美国还是中国，西方还是东方，普通人能做的只有经营生活。

十一月，我趁秋假去了芝加哥。我没有特地安排的行程，在博物馆餐吧遇到带着小孩的家长时，就多坐一会，边嚼薯片边听小人类讲恐龙和竹节虫。在密歇根湖畔坐到天色将暗，再起身步行穿过半座城市，随手拍拍身边的景色。

我穿过了一条长长的隧道，走进去很深才发现里面没有行人，只有来来往往的车。每个有暖气排放的风洞口前都躺着流浪汉，我分不清他们盖的是蓝色被子，还是建筑工地用的塑料包装。他们有的对我笑，我也回以微笑。

印象深刻的是一个车站遇到的中年人。我在去往菲尔德馆的车上坐过了头，谁知车子竟沿着公路狂飙八分钟，才来到一个枢纽站。另一位车站员工听说我要去的地方，看着我笑，“嘿，这家伙要去菲尔德！”

我之所以只能用“车站遇到的中年人”形容他，是因为他实在是和我一样的最普通人，披红色工作外套，背双肩包，没戴帽子和手套。

他反复指着不远处的车站，一遍遍叮嘱我要上哪辆车，“一定要在车子开上公路后拉绳子，下车后朝东走，穿过公园，就能到菲尔德。”在我上车后，他又站在前门口叮嘱司机，“他要去菲尔德，记得叫他下车。”我注意到他有不太整齐的牙齿，在西方文化里，这是鲜明的身份标签。

作为一个即使不饿、只要闻到食物气味、就能再吃下一碗米饭的人，我旅行的重点之一是填饱肚子。

慕名点了芝加哥深盘比萨，可惜经典配方只放番茄和芝士，我吞了两片后酸腻难忍，只好打包带走。出门时，大楼间突然飘出雪花，我和Lyft司机讲第一次来芝加哥能碰上雪真幸运，他却对我的兴奋摸不到头脑，问我是否来自从不下雪的佛罗里达。

我多想告诉他，我的家乡也有绵绵大雪，小时候总要躺在雪堆上一趟趟往下滑。

十二月，博士的第一学期结束了。我猛然意识到自己没有产出任何成果，学期初布置的很简单的课题，几乎没有任何进展。我的导师是开朗体贴、尊重学生的人，她很少催促我，还耐心地向我解释她的观点。

原来，我整套研究的方法论存在问题，而我却糊弄了一个学期。有几个夜晚焦虑爬上躯体，从实验室走回家的路上，我想多吸几口冷气，又怕走进危险街区，因此无处可去，只好在路边假装等车，样子实在滑稽。

我总处于一种慌张中，虽然日程安排不比本科时紧张，但精神上异常忙碌，甚至可以用杂乱来形容。

有段时间，我用播客塞满了所有碎片化的空闲。起床闹钟换成了Apple News，早餐和走路去工位的半小时、中午去食堂和饭后散步的一小时、晚上健身房的时间，都在耳机里放播客。疗愈、喜剧、哲学、AI、政治，杂乱无章地听。

我还买了些paperback装订书，它们在国内不常见，只有高中英语老师劳拉买给大家来读。那时读《动物农场》和《灿烂千阳》，我基本为了应付劳拉的任务来翻书，如今看来是一个遗憾。

在学校里，我幸运地找到两组中文读书会，为玉米地的生活觅得些许乐趣。一个天马行空地讲散文和诗歌，从加缪和黑客帝国，聊到科幻小说改编的街霸游戏，再聊到互联网角落里翻出的饮食博客。另一个积极关心社会现实，听许倬云说美国，又谈精英制的傲慢如何卷走了某个金色年代的辉煌承诺。

这些可爱又认真的朋友们不断提醒我，远方总是充满诱惑，但在这片宁静的玉米地，即使不能以肉身投入周边激荡的时代，我们仍能触摸和探索。

Reading Su Zizhan

Fri, 14 Nov 2025 00:00:00 +0800

苏东坡传

This is a translation by Claude. The original Chinese version is below.

I rarely read historical narratives before, but when I came across this set of “Su Dongpo” books at graduation, I decided to buy them and have a look. The complete set has three volumes, and I didn’t finish reading them before leaving the country, so I brought them all to the United States.

By now I’ve made it through two volumes, on and off, and at this point in the story Su Shi’s life is already more than half over. After the Crow Terrace Poetry Case, the man who led the Northern Song literary world, the foremost talent of his age, and who would one day serve as chancellor in a time of peace—though his heart still belonged to the nation—had already shed his long robe and headscarf for a coarse hemp jacket and straw sandals, and was out breaking new farmland in the fields of Huangzhou.

The threads of Foyin and Wu Fugu run cleverly through Su Shi’s story, as if hinting, from a later generation’s vantage point, that he was always a wanderer caught between worldly affairs and the Buddhist gate.

Chapter 48 is titled “The Small Boat Drifts Away from Here.” It opens with Emperor Shenzong’s defeat at Yongle: gravely ill, the court now run by eunuchs who extorted money and silk to buy peace with Western Xia through annual tribute, while the Northern Song sank further into decline. I thought back to when Shenzong first took the throne, brimming with ambition to accomplish great things and revive the dynasty, governing with real rigor and diligence. But the Xining Reforms failed, twenty years slipped by, and watching the dynasty’s fortunes run out, even the Son of Heaven could feel like nothing more than a grain of millet adrift on a vast sea. I found myself, unexpectedly, empathizing with the emperor.

When I was young reading history textbooks, I would look disdainfully at those emperors who indulged in wine and pleasure and neglected state affairs, scorning them as incompetent rulers. Looking back now, perhaps they had long understood the truth that “the sober are the most absurd.”

After reading just one more page, I closed the book. In my mind, I silently skipped over the dialectical discussion between Su Zi and his guest, fast-forwarding directly to the last paragraph of “Red Cliff Rhapsody”:

The guest rejoiced and laughed, rinsed the cups and poured more wine. When the food was finished and the cups and plates were in disarray, they lay down together in the boat, not knowing that dawn had broken in the east.

If life could be like this, how carefree and joyful it would be.

我此前很少读历史故事，毕业时划到这套《苏东坡》，索性买了瞧瞧。整套共三册，出国前没能读完，就一起带了美国来。

如今断断续续读了两册，行文至此，苏轼的人生已过大半。乌台诗案后，那北宋文坛领袖、当世第一才子、后世太平宰相，虽仍心系家国，却已脱下长袍头巾，披上麻布短褂，脚踩芒鞋，开荒于黄州田间了。

佛印、吴复古的脉络精巧，穿插于苏轼的人生故事中间，仿佛是从后人视角暗示，苏轼始终是世事与佛门间的游子。

第四十八章名为“小舟从此逝”，开篇讲宋神宗兵败永乐，重病不起，宦官把持朝政，搜刮钱帛供纳岁币向西夏求和，北宋日益衰落。回想起神宗登基时，满怀建功立业、振兴大宋之心，励精图治。但熙宁变法失败，二十年时光流过，眼见王朝气数将尽，纵是天子亦有渺沧海之一粟。我竟与帝王共情起来了。

小时候念历史课本，会鄙夷地看那些沉迷酒色、不理朝政的皇帝，嗤为昏君庸君，如今再看，或许他们早已参透这“清醒的人最荒唐”的道理罢。

我只又看了一页便把书合上了。在心里默默略过了苏子与客的辩证讨论，直快进到《赤壁赋》的最后一段：

客喜而笑，洗盏更酌。肴核既尽，杯盘狼藉。相与枕藉乎舟中，不知东方之既白。

人生如果如此，该有多么潇洒快活。

Sat, 25 Oct 2025 00:00:00 +0800

Stay Updated #

If you enjoy it here, subscribe to get notified about new posts.

Substack #

New posts by email — subscribe on Substack.

RSS Feed #

Copy this URL into your feed reader:

https://ray-che.com/index.xml

订阅 #

如果你喜欢这里，欢迎订阅。

最简单的方式是通过 Substack 邮件订阅。

也可以使用 RSS —— 将以下地址复制到你的 RSS 阅读器：

https://ray-che.com/index.xml

The Homeless

Fri, 24 Oct 2025 00:00:00 +0800

This is a translation by Claude. The original Chinese version is below.

Green Street is the busiest street in this small city.

Walking down Green Street, I saw something moving on the pavement in the distance—a blanket, shifting slowly. Only when I came closer did I notice her: thin and slight, her black jacket already half-swallowed by the shadows.

I didn’t dare turn back to look at her face straight on. That gesture could be read too many ways: sizing her up, pitying her, looking down on her, mocking her. Between her and everyone else passing by, there’s more than just the divide between housed and unhoused—there’s an invisible fence, a social barrier that keeps us all apart.

Last week, I saw a young white man sitting beside her at this same spot, talking with her. I wanted to hear her story too. But I’m an outsider here, drawn to this civilization with equal parts curiosity and fear.

Still, one of its symbols is known the world over. I wanted to tell her: that blanket of yours—it looks a little like Superman’s cape.

I see capes on everyone.

绿街是小城上最繁华的街道。

我走上绿街，远远地看到地上有一片毯子在挪动。走近后，我才注意到瘦窄的她。她上半身黑色的夹克衫，已有一半隐匿进背景。

我不敢回头去看她的正脸。这个动作有太多可能的解读：我打量她、我同情她、我鄙视她、我嘲笑她。在她和周围人之间，除去有房子住和没有房子住之外，另有一层社会性的围栏把人群隔离开来。

我记得上周在同样的位置，有个白人青年就靠坐在她身边，同她聊天。我也很想听她的故事，可我是文化上的外来者，对这个文明有好奇，也有恐惧。

但它的图腾之一声名远扬。我想告诉她，其实那毯子有点像超人的披风。

我在每个人身上看到披风。

Time Poverty Is the Greatest Poverty

Sun, 19 Oct 2025 00:00:00 +0800

Champaign, 2025

This is a translation by Claude. The original Chinese version is below.

I want to stop
To quietly take in the half-red maple leaves
To find a small breakfast café
And watch the world pass by the window

I could read, I could write
I could lift my camera again
Though I’ve forgotten how heavy it feels

If I had time
I could take my time
With assignments, with papers
I could see the beauty in them

But there is no time
I’m always rushing
Now I understand:
Time poverty is the greatest poverty of all

我想停下来
静静欣赏半红的枫叶
再去小小的早餐店坐下
看窗外车来车往

我可以看书写字
再拿起镜头
我已不习惯它的重量

假如有足够的时间
我就可以慢慢地完成
作业和论文
我本能欣赏它们的美

可我没有时间
我总是匆匆忙忙
原来
时间贫困是最大的贫困

My First Rain Night in the US

Fri, 05 Sep 2025 00:00:00 +0800

Champaign, 2025

This is a translation by Claude. The original Chinese version is below.

Part I #

Two weeks in Champaign, and tonight brought the first rain.

I got on the bus. Lightning kept flashing outside the window—blue streaked with orange, but no thunder. Students of all colors and hair types sat around me. Quieter than usual.

Three Asian girls were laughing together. One was teaching another Mandarin:

“My name is Sunny.” “Wo duh ming-zuh shih Sunny!” “Hahahahaha!”

I wanted to watch them longer. Their energy felt especially alive in this rural darkness.

I got off at the stop closest to my apartment. The wind nearly flipped my umbrella inside out. I had to run for cover under a nearby building entrance.

It felt like a scene from a movie. Tom Hanks clutching his suitcase, holding his hat down, ducking under an awning, muttering curses at the damn weather.

Part II #

A medium-build guy stood next to me. His lower half was soaked through. We started talking.

He studies AI philosophy—how artificial intelligence shapes human society. I quickly pulled out a respectable topic to match his, saying that whether AI has consciousness is also a profound philosophical question.

But he was more interested in sociology.

The rain lightened. I asked if he wanted me to walk him back to his place. He thanked me several times, but said just to the nearest engineering building would be fine.

So there we were, sharing an umbrella—a Brit from across the Atlantic and a Chinese guy from across the Pacific, walking side by side down a country road in the middle of America.

“I heard London rains a lot. Is that true?” “Yes, most of the time.”

Part III #

I finally told him I’d only been in the US for two weeks. This was my first rain.

He said he’d been here two years. If I ever had questions, I could ask him anytime. I fumbled through some small talk. Campus is huge. Weather’s been nice.

“But sometimes I feel lonely. Do you?” “All the time.” “How do you deal with it?” “You talk to people.”

I like these ordinary moments with ordinary people. They make me feel like I exist. Do we really only live for a few special moments? No. I don’t think so. Every moment deserves to be savored.

We exchanged numbers and wished each other a good night.

Now, as I write this, I can finally piece it all together—his British accent, the “Apologies” when the security guard turned us away, the endless “Thank you so much,” “You are so kind.”

Turns out the old-fashioned folks are still out there. The new world still has old boats to carry us.

（上） #

转眼间来香槟两周了，今天才遇到第一场雨。

我坐上公交，窗外不停有闪电划过，蓝色夹着橙色，但没有雷声。车上坐着各色皮肤和头发的同学，比往常安静些。

三个亚裔女孩说说笑笑，其中一个教另一个说普通话：

“我的名字是Sunny。” “窝的鸣自石 Sunny！” “哈哈哈哈哈！”

我很想多看她们一会。旺盛的生命活力在这大农村的夜里显得格外迷人。

我在离公寓最近的车站下了车，斜风几乎把伞吹翻，我只能慌忙地跑进旁边的公寓门口躲雨。

这一幕好像电影里的桥段。Tom Hanks提着手提箱，压着帽子狼狈地躲到屋檐下，低声咒骂着该死的天气。

（中） #

旁边站着一个中等身材的人，下半身已经湿透。我和他搭起了话。

他学习AI哲学，研究人工智能的发展对人类社会的影响。我赶忙掏出充面子的大题目，附和说研究AI到底是否拥有意识，也是非常深刻的哲学问题。

但他对社会学更感兴趣。

雨小了些，我问要不要送他回公寓。他再三表示感谢，并请我送到最近的工院就好。

就这样撑着伞，一个来自大西洋对岸的英国人和一个来自太平洋对岸的中国人，肩并肩走在美洲的乡村小路上。

“I heard London rains a lot. Is that true?” “Yes, most of the time.”

（下） #

我终于和他说起，我才刚来美国两周，这是我遇到的第一场雨。

他说自己已经待了两年，如果我有任何问题，可以随时问他。我敷衍了几句，校园好大，天气很好。

”But sometimes I feel lonely. Do you?” ”All the time.” ”How do you deal with it?” ”You talk to people.”

我喜欢这些平凡的时刻和普通的人。他们让我感受到自己的存在。人生真的只活几个瞬间吗？不，我觉得不是。每个瞬间都值得细细品味。

我们最后交换了电话号码，并相互祝福对方拥有一个美好的夜晚。

现在写字时，我才能把他的英国口音、被公寓保安拒绝时说的"Apologies"和不断对我说的"Thank you so much"、“You are so kind"串联起来。

原来，老派的人们依然存在，新世界仍有载我们的旧船。

Grade Chaser

Thu, 14 Aug 2025 00:00:00 +0800

Shenyang, 2025

This is a translation by Claude. The original Chinese version is below.

He switched his mainland number over to an 8-yuan number-keeper plan—the last item on his list before leaving the country. Tomorrow, this young grade chaser would leave home and fly to the other side of the world.

Now he could do all the things he’d dreamed of as a child. He sat behind the wheel, window rolled down all the way, arm hanging freely in the wind. He walked into McDonald’s without thinking twice, gulping down junk food. He pulled a beer from the fridge and spent the whole night gaming in his room.

He remembered graduating high school—so full of fire then, pedaling a shared bike like mad to the grandest mall in town, taking selfies, swearing the world was his for the taking.

Now the great upheaval of the era pressed down until he could hardly breathe. He knew the childhood nursery rhyme of the “global village” was on the verge of collapse, that a grain of the era’s sand had already drifted down onto him, and that he might not see home again for years.

He found himself back at his elementary school gates, at the stationery shop he’d visited every afternoon after class. The owner had gotten heavier. He didn’t dare bring up those years—just bought a roll of tape and left.

As he turned to go, he noticed the yo-yos and Rubik’s cubes still dangling by the door. His grandfather had once filled an entire drawer with them for him. Twenty years had slipped past. The person who loved him most was gone.

He’d always thought he didn’t understand this place. Didn’t even like it much. He couldn’t find his way around, didn’t know where the good restaurants were, where people went to have fun. He’d spent too much time buried in test prep—his memories were all classrooms, cram schools, and the back seat of the car shuttling between them.

But now, on the eve of leaving, tears streamed down his face without permission. Suddenly he understood those old sayings: “Objects carry memories of those we’ve lost.” “Landscapes stir the heart.” Because things and places hold our feelings, and feelings come from the people we love. As long as he carries them in his heart, his hometown will always be with him.

给国内手机号换上8元的保号套餐，他完成了出国前的最后一件事。明天，这个年轻的做题家就要离开家乡，飞往地球的另一端。

他已经可以完成很多小时候梦寐以求的事情。他坐在汽车的驾驶位，窗户可以降到最低，把手自由地伸出窗外；他随意地走进一家麦当劳，大口吃起垃圾食品；他从冰箱里掏出啤酒，回到房间打上一晚的游戏。

遥想高中毕业，他意气风发，疯也似的骑着共享单车到最繁华的商场前自拍，狂言世界就在眼前。

如今，时代的大变局压得他喘不过气。他知道，“地球村”的童谣濒临幻灭，时代的一粒沙已然飘下，自己可能几年无法回家。

他转回到小学门口，去了以前每天放学都去的文具店，店主比记忆里的发福了些。他没好意思问起当年，只匆匆买了一卷双面胶。

转头发现店门口挂着的竟然还是悠悠球和魔方，姥爷曾给他买了满满一抽屉。20年时光流过，最爱他的人已经离开了他。

他一直觉得自己不了解也不喜欢家乡。他不太认识路，也不知道哪里好吃，哪里好玩。因为花了太多时间做题，他的记忆停留在学校、补习班和往返两者之间的汽车后座。

可临别之际，他的泪水止不住地流。他恍然明白，为什么古人会"睹物思人"，“触景生情”。因为物和景是情的载体，而情是因人而生的。只要做题家的心里有人在，他与家乡的羁绊就在。

What I Learnt in Seven Years at Tsinghua

Sat, 05 Jul 2025 00:00:00 +0800

Beijing, 2025

This is a translation by Claude. The original Chinese version is below.

Tomorrow I’ll leave this school. Standing at the end of this journey, looking back at seven rushed years, I can’t help but feel many things. I’m putting it into words here, if only as encouragement—for myself, and for all of you.

I’ll never forget my undergraduate graduation ceremony three years ago. It was an unbearably hot morning. We sat crammed together on the field, wearing long-sleeved shirts and academic robes.

The student speaker took the stage. To my astonishment, she turned course material into verse with sweeping, impassioned parallelism: “Ah! In freshman calculus, we felt the harmony of [such-and-such] formula! Ah! In sophomore physics, we grasped the wonder of [such-and-such] theory! Ah! In the department building, we pulled all-nighters grinding through problem sets!”

I was stunned. At this bittersweet moment of graduation, under the blazing June sun, why on earth would she give us a final exam review? If there were a contest for “turning suffering into song,” she’d win first place.

I used to think the most important thing Tsinghua taught me was finding my gift. It’s a cliché idea with many names—“finding your lane,” “finding your passion.” What I mean is: everyone has a unique domain where things just click. In that space, you slip into flow easily. Understanding, mastery, creation—they all come naturally. You spend little time and find meaning while producing work that others struggle and strain to barely finish.

Three years later, I finally understand that graduation speaker. She’d found her gift earlier than the rest of us. And by chance, her gift aligned perfectly with the most standard, correct, proper, efficient, mainstream path.

I admire her. I wish her well. In my seven years at Tsinghua, I’ve been lucky—I’ve more or less come to know myself. I wish the same for everyone: may you find your gift soon.

明天就要离开学校，站在一段旅程的尽头，回看匆忙的七年时光，不免有诸多感慨。仅以此记录，与诸君共勉。

我永远无法忘记三年前的本科毕业典礼。那是个异常炎热的早晨，我们穿着长袖衬衫和学士服长袍，挤坐在操场。

毕业生代表登台演讲。她竟然用慷慨激昂的排比句，把专业课知识点编排成诗：“啊！我们在大一的微积分里感受xx公式的和谐！啊！我们在大二的物理课领略xx理论的魅力！啊！我们在系馆的教室里熬夜通宵刷题！”

我错愕不已，无论如何也想不通，在毕业这个悲喜交加的节点，六月的烈日当空，她为何要给大家上一堂期末复习课。如果有一场比赛，比拼“把苦难谱写成歌”，她一定荣膺最佳写手。

我以为，清华教会我最重要的一节课，是找到天赋所在。这个概念很俗套，也有很多别名，比如“找到属于自己的赛道/热爱”。我理解，每个人都有自己独一无二的天赋领域，在这个领域里，你很容易进入心流，理解、掌控和输出的能力都与生俱来；你花费少量的时间，在获得意义的同时，就能完成别人煎熬和挣扎后勉强产出的工作。

三年过后的今天，我理解了当年的毕业生代表。她比我们都提前找到了自己的天赋，而很巧，她的天赋点在最标准、正确、合规、高效、主流的路径上。

我羡慕并祝福她。在清华七年，我很幸运，也已经大概认识了自己。也祝福所有人，早日找到自己的天赋。

Not Sure if I Miss It Here

Sun, 15 Jun 2025 00:00:00 +0800

Beijing, 2025

This is a translation by Claude. The original Chinese version is below.

I found an empty classroom on the fourth floor of Building Six. Sat down. Waited for the sunset to bleed into the rain-washed sky, hoping to catch a few good shots of campus.

My first time trying the vending machine coffee. Eleven yuan. Sweet and sour and bitter all at once—oddly satisfying. In a week I’ll graduate and leave. Seven years of life coming to an end. Hard not to feel something, at twenty-five, still indecisive as ever.

I remember freshman year calculus and linear algebra were always in Building Six. Every Monday and Wednesday at 8 AM, nearly two hundred of us crammed into the basement lecture hall. The air thick with sweat, breakfast, and exhaustion. Five straight periods. Hardly anyone spoke. Just the professor flipping through slides, feeding us formula after formula. After class, we’d push through the bike lot, push down Xuetang Road, push through the dining hall, all the way back to the dorms—or off to the next classroom.

That was basically my undergrad in a nutshell. Drowning in math, physics, chemistry, coding. The intro circuits course felt impossible. Most of my academic curiosity got ground down before it ever had a chance to grow.

Do I miss this place? Of course. But what I miss isn’t the life here—it’s who I was here.

我在六教的四楼找了间教室坐下，等待夕阳混进雨后的天空，摸两张校园美景。

第一次喝自助机里的咖啡，11块钱，酸酸甜甜苦苦的，别有一份风味。一周后就要毕业离校，一段七年的生命历程即将迎来终点，难免要让优柔寡断的25岁年轻人感慨一番。

我记得大一的微积分和线性代数课总是安排在六教。每周一和周三早晨八点，近两百人就挤进负一层的阶梯教室，在弥漫着汗味、早餐味和困意的空气里，完成五节课的学习。课堂上很少人说话，基本是老师在不停地翻课件，把一行行公式塞给我们。下课之后，大家在停车场挤，在学堂路挤，在食堂挤，一路挤回宿舍，或者赶去下一间教室。

这几乎是我本科时代的缩影。我在数学物理化学编程课里疲于奔命，电路入门课难度更犹如登天，让我的大部分学术兴趣在入门时就被繁重的课程消耗殆尽。

我怀念这里吗？当然。但我怀念的大概不是这里的生活，而是这里的自己。

Relearning C++

Sun, 23 Feb 2025 00:00:00 +0800

Lately, I’ve been using LeetCode to prepare for an internship and was surprised by how much the C++ standard has evolved since I studied it as an undergraduate. It’s fascinating to revisit C++ and explore the new features and improvements.

Data Structures #

Vector #

vector is a sequence container in the C++ Standard Library that provides dynamic array functionality.

#include <vector>
using namespace std;
vector<int> numbers = {1, 2, 3};

// as function parameter
vector<int> twoSum(vector<int>& nums, int target) {};

// access element
cout << "The first element of the vector is:" << numbers[0] << endl;

// add element
numbers.push_back(1);
numbers.push_back("hello");

// remove element 
numbers.pop_back();

// used in loops
for (int num : numbers) {};

// size
int size = numbers.size();

// check if it's empty
bool isEmpty = numbers.empty();

// sum
int totalSum = accumulate(numbers.begin(), numbers.end(), 0);

String #

#include <string>
using namespace std;
string str = "I love engineering."

// length
int len = str.length();

// used in loops
for (char c : str) {};

// find substr
bool isSubstr = str.find("love")

// return the starting index of the substr
int SubstrIdx = str.find("love")

// extract a substring
string substr = str.substr(pos, len)

// check if it's empty
bool isEmpty = str.empty();

Unordered_map #

unordered_map is a type of associative container in the C++ Standard Library used to store key-value pairs.

#include <unordered_map>
using namespace std;
unordered_map<char, int> myMap = {
            {'I', 1},
            {'V', 5}
        };

// Checking if a Key Exists:
if (myMap.find(3) != myMap.end()) {
    // Key 3 exists
}

Hello World

Tue, 14 Jan 2025 00:00:00 +0800

Hello World!

你好！

Home on Xijia Che

Confirming Claims of Superposition and Adversarial Examples in Toy Models

tl;dr #

Background #

Toy Model and Notation #

Attacks Exploit Interference #

No Superposition, Little Vulnerability #

Shared Geometry Drives Transfer #

Beyond Replication: Orthogonalize or Collapse? #

Appendix A #

The Boundless Voice of Night

My First Semester of a PhD in America

Reading Su Zizhan

Subscribe

Stay Updated #

Substack #

RSS Feed #

订阅 #

The Homeless

Time Poverty Is the Greatest Poverty

My First Rain Night in the US

Part I #

Part II #

Part III #

（上） #

（中） #

（下） #

Grade Chaser

What I Learnt in Seven Years at Tsinghua

Not Sure if I Miss It Here

Relearning C++

Data Structures #

Vector #

String #

Unordered_map #

Hello World