I work for Amazon as a software engineer, and primarily work on a mixture of LLM’s and compositional models. I work mostly with scientists and legal entities to ensure that we are able to reduce our footprint of invalid data (i.e. anything that includes deleted customer data, anything that is blocked online, things that are blocked in specific countries, etc). It’s basically data prep for training and evaluation, alongside in-model validation for specific patterns that indicate a model contains data it shouldn’t have (and then releasing a model that doesn’t have that data within a tight ETA).
It can be interesting at times, but the genuinely interesting work seems to happen on the science side of things. They do some cool stuff, but have their own battles to fight.
That sounds cool, I’ve had roles that were heavy on data cleansing, although never on something so interesting. What languages / frameworks are used for transforming the data, I understand if you can’t go into too much detail.
I did wonder how much software engineers contribute in the field, it’s the scientists doing the really interesting stuff when it comes to AI? Not surprisingly I guess 😂
I’m a full stack engineer, I was thinking of getting into contracting, now I’m not so sure, I don’t know enough about AI’s potential coding capabilities to know whether I should be concerned about job security in the short, or long term.
Getting involved in AI in some capacity seems like a smart move though…
We do a lot of orchestration of closed environments, so that we can access critical data without worry of leaks. We use Spark and Scala for most of our applications, with step functions and custom EC2 instances to host our environments. This way, we build verticals that can scale with the amount of data we process.
If I’m perfectly honest, I don’t know how smart a move it is, considering our org just went through layoffs. We’re popular right now, but who knows how long for.
It can be interesting at times, but to be honest if I were really interested in it, I would go back and get my PhD so I could actually contribute. Sometimes, it feels like SWE’s are support roles, and science managers only really care that we are unblocking scientists from their work. They rarely give a shit if we release anything cool.
EnderMB@lemmy.world 10 months ago
I work for Amazon as a software engineer, and primarily work on a mixture of LLM’s and compositional models. I work mostly with scientists and legal entities to ensure that we are able to reduce our footprint of invalid data (i.e. anything that includes deleted customer data, anything that is blocked online, things that are blocked in specific countries, etc). It’s basically data prep for training and evaluation, alongside in-model validation for specific patterns that indicate a model contains data it shouldn’t have (and then releasing a model that doesn’t have that data within a tight ETA).
It can be interesting at times, but the genuinely interesting work seems to happen on the science side of things. They do some cool stuff, but have their own battles to fight.
krazzyk@lemmy.world 10 months ago
That sounds cool, I’ve had roles that were heavy on data cleansing, although never on something so interesting. What languages / frameworks are used for transforming the data, I understand if you can’t go into too much detail.
I did wonder how much software engineers contribute in the field, it’s the scientists doing the really interesting stuff when it comes to AI? Not surprisingly I guess 😂
I’m a full stack engineer, I was thinking of getting into contracting, now I’m not so sure, I don’t know enough about AI’s potential coding capabilities to know whether I should be concerned about job security in the short, or long term.
Getting involved in AI in some capacity seems like a smart move though…
EnderMB@lemmy.world 10 months ago
We do a lot of orchestration of closed environments, so that we can access critical data without worry of leaks. We use Spark and Scala for most of our applications, with step functions and custom EC2 instances to host our environments. This way, we build verticals that can scale with the amount of data we process.
If I’m perfectly honest, I don’t know how smart a move it is, considering our org just went through layoffs. We’re popular right now, but who knows how long for.
It can be interesting at times, but to be honest if I were really interested in it, I would go back and get my PhD so I could actually contribute. Sometimes, it feels like SWE’s are support roles, and science managers only really care that we are unblocking scientists from their work. They rarely give a shit if we release anything cool.